atom feed10 messages in com.googlegroups.protobuf[protobuf] Re: Serialize with Length ...
FromSent OnAttachments
Jay ThomasAug 24, 2009 3:54 pm 
Peter KeenAug 24, 2009 4:05 pm 
Jay ThomasAug 24, 2009 4:07 pm 
Jay ThomasAug 24, 2009 4:17 pm 
Peter KeenAug 24, 2009 4:24 pm 
Peter KeenAug 24, 2009 4:33 pm 
Jay ThomasAug 24, 2009 4:42 pm 
Marc GravellAug 25, 2009 12:07 am 
Paul ShaferFeb 20, 2013 1:32 pm 
Marc GravellFeb 20, 2013 8:57 pm 
Subject:[protobuf] Re: Serialize with Length Prefix
From:Paul Shafer (prab@gmail.com)
Date:Feb 20, 2013 1:32:29 pm
List:com.googlegroups.protobuf

First time poster and new to protobuf/protobuf-net so forgive me if this has already been addressed. But I've searched quite a bit and cant find an"official" answer.

If you want to use the fieldNumber argument in your SerializeWithLengthPrefix/TryDeserializeWithLengthPrefix for message type identification (or for whatever reason), all works great between two C#/.NET apps. However if you have a c++ app using protobuf talking to a C#/.NET app using protobuf-net, the fieldNumber argument will cause grief. It took wireshark, plus diving into protobuf-net code and protobuf code to find that the fieldNumber is encoded like (I assume) any other field within a message. Whereas the c++ protobuf has no such notion (that I could find) unless you want to use a CodedOutputSTream - which then has both a WriteTag and WriteVarint32 (which WriteTag calls). However WriteTag/WriteVarint32 does NOT encode the same as the fieldNumber with SerializeWithLengthPrefix. So... what do to?

Again, perhaps this has been addressed already but here's my solution (on the c++ side):

#define PROTO_NET_TAG(x) (((x) << 3) | (2 & 7))

... google::protobuf::io::ZeroCopyOutputStream* raw_output = new google::protobuf::io::StringOutputStream(str); google::protobuf::io::CodedOutputStream * coded = new google::protobuf::io::CodedOutputStream(raw_output); coded->WriteTag(PROTO_NET_TAG(msgTag)); coded->WriteVarint32(msg->ByteSize()); msg->SerializeToCodedStream(coded);

On Monday, August 24, 2009 5:33:26 PM UTC-6, Peter Keen wrote:

I looked at the protobuf-net source a little bit and it looks like if you write the length with WriteLittleEndian32 instead of WriteVarint32 you should be good to go, as long as you use PrefixStyle.Fixed32 within the C# call to DeserialzeWithLengthPrefix().

--Pete

On Mon, Aug 24, 2009 at 4:17 PM, Jay Thomas<jayd@gmail.com<javascript:>> wrote:

This code ideally has to operate with C# code on the other side of the socket that uses the DeserializeWithLengthPrefix() method call. The message originates in C++ environment on a linux box and terminates in C# environment on a windows PC. Why doesn't C++ support the same methods as C#? The only document that I've seen that has list of methods available is the message.h header file, installed into my Linux usr directory.

On Aug 24, 4:05 pm, Peter Keen <pete@gmail.com> wrote:

You sort of have to roll your own. In my project I'm doing something like this:

coded_output->WriteVarint32(message->ByteSize()); message->SerializeToCodedStream(coded_output);

And then on the reading side:

uint32_t size; if (! coded_input->ReadVarint32(&size)) { return NULL; // just an example. probably don't want to do this;

}

Message * m = new Message; CodedInputStream::Limit limit = coded_input->PushLimit(size); message->ParseFromCodedStream(coded_input); coded_input->PopLimit(limit);

The biggest thing that helped me along was finding the Limit docs. I couldn't figure out how to parse from a ZeroCopyInputStream without consuming the entire stream, and then I ran across that and everything became much more clear.

--Pete

On Mon, Aug 24, 2009 at 3:54 PM, Jay Thomas<jayd@gmail.com> wrote:

Hello

I am looking for a way to serialize/deserialize with length prefix under c++. The serialized bytes will sent to a TCP socket. I am aware that C# has a method SerializeWithLengthPrefix() and DeserializeWithLengthPrefix(). Are there any such analogous methods for C++? Please point out any documentation that I may have missed here.

Thanks for any assistance.

Jay