Google Protocol Buffer (Protobuf for short) is a lightweight and efficient structured data storage format that is platform-independent, language-independent, and extensible. It can be used in fields such as communication protocols and data storage.
Data interaction xml, json, protobuf format comparison
1, json : In general web projects, json is the most popular. Because the browser supports json data very well, there are many built-in function support.
2, xml: is the most widely used in webservice, but compared to json, its data is more redundant, because it requires a pair of closed tags. json uses key-value pairs, which not only compresses a certain amount of data space, but is also readable.
3. Protobuf: a rising star, it is a data format open sourced by Google, suitable for high-performance data transmission scenarios that require response speed. Because profobuf is a binary data format, encoding and decoding are required. The data itself is not readable. Therefore, only real readable data can be obtained after deserialization. Compared with other protobuf, it has more advantages
1: The volume after serialization is smaller than Json and XML, suitable for network transmission
2: Supports cross-platform and multi-language
3: Message format upgrade and compatibility are not bad
4: Serialization and deserialization Very fast, faster than Json processing speed
Advantages of protoBuf
Protobuf is like XML, but it Smaller, faster, and simpler. You can define your own data structure, and then use the code generated by the code generator to read and write this data structure. You can even update the data structure without redeploying the program. Just use Protobuf to describe the data structure once, and you can easily read and write your structured data in a variety of different languages or from a variety of different data streams. It has a great feature, that is, “backwards” compatibility is good, people can upgrade the data structure without destroying the deployed programs that rely on the “old” data format.
Protobuf has clearer semantics and does not need something like an XML parser (because the Protobuf compiler will compile .proto files to generate corresponding data access classes to serialize and deserialize Protobuf data). There is no need to learn a complex document object model to use Protobuf.
The programming model of Protobuf is friendly, simple and easy to learn. At the same time, it has good documentation and examples. For people who like simple things, Protobuf is better than other technologies. More attractive.
The shortcomings of ProtoBuf
Protobuf also has shortcomings compared to XML . It has simple functions and cannot be used to express complex concepts. XML has become a standard authoring tool for many industries, and Protobuf is only a tool used internally by Google, which is far less versatile in terms of versatility. Since text is not suitable for describing data structure, Protobuf is not suitable for modeling text-based markup documents (such as HTML). In addition, because XML is self-explanatory to a certain extent, it can be directly read and edited by people. At this point, Protobuf is not good. It is stored in a binary format. Unless you have a .proto definition, you cannot read it directly. Any content from Protobuf
Protobuf installation
Install protoBuf
#Download protoBuf:
$ git clone https://github.com/protocolbuffers/protobuf .git
#Install dependent libraries
$ sudo apt-get install autoconf automake libtool curl make g++ unzip libffidev
-y
#Install
$ cd protobuf/
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install
$ sudo ldconfig # Refresh shared library
#Required after success Use command to test
$ protoc --h
Get the proto package
#Go language proto API interface
$ go get -v -u github.com/golang/protobuf/proto
Install the protoc-gen-go plugin
#Install
$ go get -v -u github.com/golang/protobuf/protoc-gen-go
#Compile
$ cd $GOPATH/src/github.com/golang/protobuf/protoc-gen-go/
$ go build
#Place the generated protoc-gen-go executable file in the /bin directory
$ sudo cp protoc-gen-go /bin/
The syntax of protobuf
Define a message type
syntax = "proto3";
message PandaRequest {
string name = 1;
int32 shengao = 2;
repeated int32 tizhong = 3;
}
The PandaRequest message format has 3 fields, and the data carried in the message corresponds to each field. Each of these fields has a name and a type.
The first line of the file specifies that you are using proto3 syntax: if you do not specify this, the compiler will use proto2. The specified syntax line must be the first non-empty non-comment line of the file.
In the above example, all fields are of scalar type: two integer types (shengao and tizhong) and one string type (name).
Repeated keyword means repeated, so in go language, use slice to represent. Just like the above file format, in the message definition, each field has a unique identifier.
Add more message types
Multiple message types can be defined in a .proto file. This is particularly useful when defining multiple related messages-for example, if you want to define the response message format corresponding to the SearchResponse message type, you can add it to the same .proto file
syntax = "proto3";
message PandaRequest {
string name = 1;
int32 shengao = 2;
int32 tizhong = 3;
}
message PandaResponse {
...
}
Add notes
To add comments to the proto file, you can use the C/C++/java/Go style double slash (//) syntax format, such as:
syntax = "proto3";
message PandaRequest {
string name = 1; //Name
int32 shengao = 2; //Height
int32 tizhong = 3; //Weight
}
message PandaResponse {
...
}
What is generated from the .proto file?
When the protocol buffer compiler is used to run the .proto file, the compiler will generate code in the selected language, which can manipulate the messages defined in the .proto file Types include getting and setting field values, serializing messages into an output stream, and parsing messages from an input stream.
For C++, the compiler will generate a .h file and a .cc file for each .proto file. Each message in the .proto file has a corresponding class.
For Python, it’s a bit different-the Python compiler generates a module containing a static descriptor for each message type in the .proto file, which is associated with a metaclass At runtime (runtime) is used to create the required Python data access classes.
For go, the compiler generates a .pd.go file for each message type.
Standard data type
A scalar message field can contain one of the following types-this table shows the types defined in the .proto file, and the corresponding ones defined in the automatically generated access class Type:
default value
When a message is parsed If the encoded information does not contain a specific element, the field corresponding to the parsed object lock is set to a default value, which is specified as follows for different types:
For strings, the default is an empty string< /p>
For bytes, the default is an empty bytes
For bools, the default is false
For numeric types, the default is 0
Use other message types
You can use other message types as field types. For example, assuming that each PersonInfo message contains a Person message, you can define a Result message type in the same .proto file, and then specify a Person type field in the PersonInfo message
message PersonInfo {
repeated Person info = 1;
}
message Person {
string name = 1;
int32 shengao = 2;
repeated int32 tizhong = 3;
}
Use proto2 message type
In your proto3 message It is also possible to import the message type of proto2, and vice versa, and then the proto2 enumeration cannot be used directly in the identifier of proto3 (it is possible if it is only used in the proto2 message).
Nested type
You can define, Use the message type. In the following example, the Person message is defined in the PersonInfo message, such as:
message PersonInfo {
message Person {
string name = 1;
int32 shengao = 2;
repeated int32 tizhong = 3;
}
repeated Person info = 1;
}
If you want to reuse this message type outside of its parent message type, you need to use it in the form of PersonInfo.Person, such as:
message PersonMessage {
PersonInfo.Person info = 1;
}
Of course, you can also nest messages to any level, such as:
message Grandpa {// Level 0
message Father {// Level 1
message son {// Level 2
string name = 1;
int32 age = 2;
}
}
message Uncle {// Level 1
message Son {// Level 2
string name = 1;
int32 age = 2;
}
}
Define Service (Service)
If you want to use the message type in the RPC (remote method call) system, you can define an RPC service interface in the .proto file, and the protocol buffer compiler will vary according to the selection Language generation service interface code and stubs. For example, if you want to define an RPC service and have a method that can receive SearchRequest and return a SearchResponse, you can define the following in the .proto file:
service SearchService {
//rpc service function name (incoming parameters) return ( Return parameters)
rpc Search (SearchRequest) returns (SearchResponse);
}
Demo:
syntax = "proto2";
package proto.client_type;
import "proto_basic.proto";
option java_package = "com.xxxx.common.proto";
option java_outer_classname = "ClientTypeProto";
message ClientTypeInfo {
optional uint32 client_type_id = 1; // Type ID
optional string client_type_code = 2; // Type identification
optional string client_type_name = 3; // Type name
optional string create_time = 4; // Creation time
optional string update_time = 5; // Update time
}
//Get terminal type list parameters
message QueryClientTypeListRequest {
optional proto.basic.RequestBasic basic = 1;
optional string client_type_code = 2; // Type identification
optional string client_type_name = 3; // Type name
optional uint32 index = 4; // Page number
optional uint32 size = 5; // Single page length
}
//Return to terminal type list parameters
message QueryClientTypeListResponse {
optional proto.basic.ResponseBasic basic = 1;
optional uint32 current_page = 2; // Current page number
optional uint64 total_page = 3; // Total page number
optional uint64 total_count = 4; // Total number
repeated ClientTypeInfo client_type_list = 5; //Event Information
}
Generate proto
protoc --java_out=../java proto_upush_wx_account.proto
# Download protoBuf:
$ git clone https://github.com/protocolbuffers/protobuf .git
#Install dependent libraries
$ sudo apt-get install autoconf automake libtool curl make g++ unzip libffidev
-y
#Install
$ cd protobuf/
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install
$ sudo ldconfig # Refresh shared library
#You need to use command to test after success
$ protoc –h
#Go language proto API interface
$ go get -v -u github.com/golang/protobuf/proto
#Install
$ go get -v -u github.com/golang/protobuf/protoc-gen-go
#Compile
$ cd $GOPATH/src/github.com/golang/protobuf/protoc-gen-go/
$ go build
#Place the generated protoc-gen-go executable file in the /bin directory
$ sudo cp protoc-gen-go /bin/
syntax = " span>proto3";
message PandaRequest {
string name = 1;
int32 shengao = 2;
repeated int32 tizhong = 3;
}
syntax = "proto3";
message PandaRequest {
string name = 1;
int32 shengao = 2;
int32 tizhong = 3;
}
message PandaResponse {
...
}
syntax = "proto3";
message PandaRequest {
string name = 1; //Name
int32 shengao = 2; //Height
int32 tizhong = 3; //Weight
}
message PandaResponse {
...
}
message PersonInfo {
repeated Person info = 1;
}
message Person {
string name = 1;
int32 shengao = 2;
repeated int32 tizhong = 3;
}
message PersonInfo {
message Person {
string name = 1;
int32 shengao = 2;
repeated int32 tizhong = 3;
}
repeated Person info = 1;
}
message PersonMessage {
PersonInfo.Person info = 1;
}
message Grandpa {// Level 0
message Father {// Level 1
message son {// Level 2
string name = 1;
int32 age = 2;
}
}
message Uncle {// Level 1
message Son {// Level 2
string name = 1;
int32 age = 2;
}
}
service SearchService {
//rpc service function name (incoming parameters) return ( Return parameters)
rpc Search (SearchRequest) returns (SearchResponse);
}
syntax = "proto2";
package proto.client_type;
import "proto_basic.proto";
option java_package = "com.xxxx.common.proto";
option java_outer_classname = "ClientTypeProto";
message ClientTypeInfo {
optional uint32 client_type_id = 1; // Type ID
optional string client_type_code = 2; // Type identification
optional string client_type_name = 3; // Type name
optional string create_time = 4; // Creation time
optional string update_time = 5; // Update time
}
//Get terminal type list parameters
message QueryClientTypeListRequest {
optional proto.basic.RequestBasic basic = 1;
optional string client_type_code = 2; // Type identification
optional string client_type_name = 3; // Type name
optional uint32 index = 4; // Page number
optional uint32 size = 5; // Single page length
}
//Return to terminal type list parameters
message QueryClientTypeListResponse {
optional proto.basic.ResponseBasic basic = 1;
optional uint32 current_page = 2; // Current page number
optional uint64 total_page = 3; // Total page number
optional uint64 total_count = 4; // Total number
repeated ClientTypeInfo client_type_list = 5; //Event Information
}
protoc --java_out=../java proto_upush_wx_account.proto