Sep 15 2014 |
First prototype - then named writer_encoder - was written while I was on a plane with no wifi.
|
Oct 27 2014 |
Rename to bincode
|
Apr 05 2015 | Initial port to serde written by erickt Prior to this, we were using rustc-serialize |
Aug 08 2015 |
Servo starts using bincode in ipc-channel
|
Jan 12 2016 | Tarpc starts using bincode |
Apr 21 2017 | Serde hits 1.0! Serde maintainer dtolnay ports bincode to use the new Serde APIs |
At it’s root, Bincode is a serializer implementation for Serde. If you stick a #[derive(Deserialize, Serialize)]
on your struct, Bincode can efficiently serialize and deserialize those structs to and from bytes.
If Bincode is just another serializer implementation, what sets it apart from all the others?
Bincode is unique in that it’s a format that was built specifically for the Rust serialization ecosystem. Tight coupling with Serde allows Bincode to be very fast and serialize to very small payloads.
Bincode achieves its speed and size wins is by not encoding structure information into the serialized output. It relies on the fact that #[derive(Deserialize, Serialize)]
implementations of the Serde traits deserializes fields in exactly the same order as it serialized them. (If you implement those traits by hand, you need to uphold this invariant as well.)
Because Bincode leaves out this extraneous structure information, a struct serialized with Bincode is often smaller than it was in memory! Let’s take a look at how Bincode serializes some common structures.
All the Rust numbers are encoded directly into the output in little-endian format by default.
use bincode::{serialize, deserialize};
let bytes: Vec<u8> = serialize(&123456789u32)?;
// a 4-byte u32 gets serialized to 4 bytes.
assert_eq!(bytes.len(), 4);
let number: u32 = deserialize(&bytes)?;
Strings are serialized by first serializing the length and then serializing the byte content of the string.
use bincode::{serialize, deserialize};
let bytes: Vec<u8> = serialize(&String::from("hello!"))?;
let string: String = deserialize(&bytes)?;
When serializing a struct, each field is serialized in order of its declaration in the struct. No additional field information is encoded.
use bincode::{serialize, deserialize};
#[derive(Serialize, Deserialize)]
struct Person {
age: u32,
name: String
}
let person = Person {
// hey, making these svgs is hard, I'm reusing values for my own sanity.
age: 123456789,
name: String::from("hello!"),
};
let bytes: Vec<u8> = serialize(&person)?;
let person_2: Person = deserialize(&bytes)?;
Enums are serialized as a tag (u32) followed by their fields serialized in declaration order.
use bincode::{serialize, deserialize};
#[derive(Serialize, Deserialize)]
struct NumberOrString {
Number(u32),
String(String),
}
let num = NumberOrString::Number(123456789);
let string = NumberOrString::String("hello!");
let bytes_num: Vec<u8> = serialize(&num)?;
let bytes_string: Vec<u8> = serialize(&string)?;
let num_out: NumberOrString = deserialize(&bytes_num)?;
let num_out: NumberOrString = deserialize(&bytes_string)?;
Bincode is great for very specific serialization tasks, but is less than ideal for others. To help you decide if you should use it or not, I’ve provided a helpful cost benefit analysis below.
Pros
Cons
The “no structure changes” drawback can not be understated. If your program requires backwards-compatible data representations, take a look at other formats such as ProtoBuf or Cap’n Proto.
However, there are many areas where those limitations aren’t actual issues! The best example of this is ipc-channel which uses bincode to send structs across the process boundary. When both processes are the same binary, there’s no need to worry about the struct definitions being different.
Another popular use case would be video games. Networked video games rarely permit players of different versions of a game to connect to each other. Similarly to the ipc-channel example, if every player is running the same build, there is no need to worry about back-compat issues.
Bincode 1.0.0 is at a point where I feel comfortable recommending the project and am happy overall with the library ergonomics.
Being both a library and an ad-hoc data format, Bincode has some interesting compatibility requirements.
Both of these are fairly easy to achieve while also permitting Bincode to evolve through the use of configuration options. As new language features come online (I’m looking at you impl-trait), bincode will be re-released with major version changes.