tars Call Chain

Problem Background

When a production system faces true high concurrency, or is decoupled into a large number of microservices, the key tasks that were previously easy to implement become difficult, and the call of each component in the distributed system needs to be analyzed. In this context, OpenTracing proposes a platform-independent, vendor-independent distributed tracking system standard semantic model and standard API. With the principle of embracing open source, it has given priority to supporting this set of solutions that are becoming the global distributed tracing unified concept and data standard.

Design method

Data collection

The tars framework internally uses the opentracing api to encapsulate the call chain data collection function and encapsulates the zipkin-brave implementation of the opentracing api.

Data tracking

Data sampling

Report content

The span and span context information generated by each kind of data track is defined in the opentracing standard definition.

Client custom report

Users can report their own defined events and information in the business code through the custom interface provided by the framework.

Data reporting

Data reporting format

Data format

The reported data format is related to the specific call chain data collection instance. Different instances have different formats. The following is an example of quick integration of zipkin to demonstrate a strongly readable json format.

json format

[
	{
	"traceId": "16661f6cb5d58903", #trace id, required
	"id": "16661f6cb5d58903", #span id，required
	"parentId": "89e051d5394b90b1", #parent span id，optional
	"king": "client",
	"name": "encode", #service function name
	"timestamp": 1510043590522358, #Initial time stamp
	"duration": 499867, #Span time, end timestamp minus start timestamp
	"localEndpoint": { #Main call information
			"serviceName": "tracer‐demo",
			"ipv4": "192.168.99.1",
			"port": 9090
	},
	"remoteEndpoint": {#information of which is called
			"serviceName": "tracer‐demo",
			"ipv4": "192.168.99.1",
			"port": 9090
	},
	"annotations": [ #span log，the events of cs, cr, sr, ss are all included
	{
			"value": "", #Transform the events except event to strings like key key1=value1 key2=value2.
			"timestamp": ""
	}
]
"tags": { #custom tag
	"${name of key}": "value",
	...
}
	"debug": true,
	"shared" true
}
]

Data analysis and display

The integrated zipkin system is built in the framework, users can query call chain data on the zipkin page.

Tars-java Usage

At present, tars-java supports the call chain from version 1.6.1, and the cpp version is about to be released. The following is an example of integrated zipkin.

Enable the call chain

To enable the call chain function, user do not need to rewrite a line of business code, just add the specified service configuration.

Description of the operation steps:

  1. On the tars management platform, select the service which need to open the call chain and click “update”.

    1

2.Select and edit the private template and fill the following:

<tars>
	<application>
		<server>
			samplerate=1   
			sampleaddress=http://localhost:9411/  
			sampletype=http  
			sampleencoding=json
		</server>
	</application>
</tars>

Illustration:

Parameter name Type Eligible Value Range Remarks
samplerate Float [0,1] Sampling rate: ranging from 0 to 1. 0 means not sampling, sampling by percentage.
sampleaddress String None Report address of sampled data: If using the http mode, fill the url of the data collection web server, if using zipkin to collect the data, fill the url of the zipkin server. If the kafka method is used for reporting, fill the address of the kafka cluster.
sampletype String http, kafka, kafka08 Sampling data reporting transmission mode: for the implementation of integrated zipkin, currently supports two transmission modes through http and kafka. Filling in with “http” means using the http protocol to report, and filling in with “kafka” means transferring the collected data to kafka. The “kafka08” option is for compatibility with kafka brokers below 0.10 version. For details, see the first note below.
sampleencodeing String json, proto Encoding format of sampling data report transmission: For the implementation of integrated zipkin, json and protocol buffer proto3 are supported. Filling in with “json” means using json encoding, and filling in with “proto” means using pb encoding.

Notes:

Custom report information:

Tars-java allows users to report custom information in the business code to the call chain tracking system, which can be completed by calling the API provided by the framework. The usage examples are as follows:

Report key-value pair information:

TraceContext.getInstance().tag("key", "value");

Key, value is a custom value.

Report event information:

TraceContext.getInstance().log("an event has occurred");

Message content is custom content

Call chain display

Now take the integrated zipkin as an example to show the reported data.

2

3

Click on a single call chain to view details:

5

Call chain key-value information description:

key value
client.ipv4 client ipv4 address.
client.port client port.
server.ipv4 server ipv4 address.
server.port server port.
tars.interface server is called the interface name (or an obj for tars-java).
tars.method server is called the method name.
tars.protocol data encoding protocol.
tars.retcode return code of the call, 0 for success and non-zero for failure.
tars.client.version client tars version.
tars.server.version server tars version.

Dependency analysis display:

6