Delta

From Freeciv

Contents 1 Delta 2 Compression 3 Files 4 Changing the definition file 5 Capabilities and variants

[edit] Delta

If delta is enable for this packet the packet-payload (after the 3 byte packet-header) is followed by the delta-header. The delta-header is a bitvector which represents all non-key fields of the packet. If the field has changed the corresponding bit is set and the field value is so included in delta-body. The values of the unchanged fields will be filled in from an old version at the receiving side.

For bool field another optimization called bool-header-folding is applied. Instead of sending an indicator in the bitvector if the given bool values has changed (and so using 1 byte for the real value) the actual value of the bool is transfered in the bitvector bit of this bool field.

Another optimization called array-diff is used to reduce the amount of elements transfered if an array is changed. This is independent of the delta-header bit i.e. it will only be used if the array has changed its value and the bit indicates this. Instead of transferring the whole array only a list of (index,new value of this index) pairs are transferred. The index is 8bit and the end of this pair list is denoted by an index of 255.

[edit] Compression

To further reduce the network traffic the (delta) packets are compressed. To get better compression results multiple packets are grouped together and compressed into a chunk. This chunk is then transfered as a normal packet. A chunk packet start with the 2 byte length field which every packet has. A chunk packet has no type. A chunk packet is identified by having a too large length field. If the length of the packet is over COMPRESSION_BORDER it is a chunk packet. It will be uncompressed at the receiving side and re-feed into the receiving queue.

If the length of the chunk packet can't be expressed in the available space of the 16bit length field (>48kb) the chunk is sent as a jumbo packet. The difference between a normal chunk packet and a jumbo chunk packet is that the jumbo packet has JUMBO_SIZE in the size field and has an additional 4 byte len field after the 2 byte len field. The second len field contains the the size of the whole packet (2 byte first length field + 4 byte second length field + compressed data).

Packets are grouped for the compression based on the PACKET_PROCESSING_STARTED/PACKET_PROCESSING_FINISHED and PACKET_FREEZE_HINT/PACKET_THAW_HINT packet pairs. If the first (freeze) packet is encountered the packets till the second (thaw) packet are put into a queue. This queue is then compressed and sent as a chunk packet. If the compression would expand in size the queued packets are sent uncompressed as "normal" packets.

The compression level can be controlled by the FREECIV_COMPRESSION_LEVEL environment variable.

[edit] Files

There are four file/filesets involved in the delta protocol:

1. the definition file (common/packets.def).
2. the generator (common/generate_packets.py).
3. the generated files (*/*_gen.[ch] or as a list client/civclient_gen.c, client/packhand_gen.h, common/packets_gen.c, common/packets_gen.h, server/hand_gen.h and server/srv_main_gen.c).
4. the overview (README.delta, this file)

The definition file lists all valid packet types with their fields. The generator takes this as input and creates the generated files.

For adding and/or removing packets and/or fields you only have to touch the definition file. If you however plan to change the generated code (adding more statistics for example) you have to change the generator.

[edit] Changing the definition file

Adding a packet:

1. choose an unused packet number. The generator will make sure that you don't use the same number two times.
2. choose a packet name. It should follow the naming style of the other packets: PACKET_<group>_<remaining>. <group> may be SERVER, CITY, UNIT, PLAYER, DIPLOMACY and so on.
3. decide if this packet goes from server to client or client to server
4. choose the field names and types
5. choose packet and field flags
6. write the entry into the corresponding section of common/packets.def

If you add a field which is a struct (say "foobar") you have to write the following functions: dio_get_foobar, dio_put_foobar and are_foobars_equal.

Removing a packet:

1. add a mandatory capability
2. remove the entry from common/packets.def

Adding a field:

Option A:

1. add a mandatory capability
2. add a normal field line:

COORD x

Option B:

1. add a non-mandatory capability (say "new_version")
2. add a normal field line containing this capability in an add-cap flag:

COORD x; add-cap(new_version)

Removing a field:

Option A:

1. add a mandatory capability
2. remove the corresponding field line

Option B:

1. add a non-mandatory capability (say "cleanup")
2. add to the corresponding field line a remove-cap flag

After changing the definition file the generator has to be run. The common/Makefile will take care of this. You don't need to run autoconf/automake/configure.

[edit] Capabilities and variants

The generator has to generate code which supports different capabilities at runtime according to the specification given in the definitions with add-cap and remove-cap. The generator will find the set of used capabilities for a given packet. Lets say there are two fields with "add-cap(cap1)" and one field with an "remove-cap(cap2)" flag. So the set of capabilities are cap1, cap2. At runtime the generated code may run under 4 different capabilities:

• neither cap1 nor cap2 are set
• cap1 is set but cap2 isn't
• cap1 is not set but cap2 is
• cap1 and cap2 are set

Each of these combinations is called a variant. If n is the number of capabilities used by the packet the number of variants is 2^n.

For each of these variant a seperate send and receive function will be generated. The variant for a packet and a connection are calculated once and then saved in the connection struct.