Skip to main content

RelAccX class

RelAccX is MistServer's reliable access class. It provides reliable access to memory data structures across multiple processes and/or threads.

The static offsets mean that writing and reading can happen simultaneously and independently, and assuming write sizes are atomic (native integer width or smaller), their reads will be atomic as well. Bigger writes are non-atomic, and will need some other form of protection against partial reads.

This means that the class is only fully reliable/safe in 32-bit (and higher) systems, since the fields used for access are all 32 bits wide.

The offsets are stored in an index at the front of the data, allowing for forwards and backwards compatibility between versions that have more or less fields than the previous/next version. Attempting to access a non-existent field will return a null pointer, while unsupported fields can simply be ignored.

Normally, the data is present on a shared memory page. These are cross-platform IPC storage primitives that allow sharing arbitrary memory pointers of pre-set sizes between processes and threads. If sharing of data is only intended to happen between threads, it may also be any other kind of shared memory access (e.g. a malloc'ed pointer).

Flags​

There are three header flags in RelAccX structures:

The ready flag is set to true as soon as the headers have been fully written. Reads are forbidden unless this flag is set. The flag may never be unset afterwards. Once set, the field_offset, record_size, record_offset and field descriptions may no longer change. record_count may still go up (but never down), unless records_deleted is non-zero (in which case it may not change at all).

The exit flag is set to true if the structure is abandoned by its writer. Its exact meaning is left up to the context, but it generally will mean that the page should be closed by all readers.

The reload flag assists with updating the structure itself. Once set it may never be unset. It signals users of the page that the shared memory page should be considered outdated and must be closed and re-opened. The intent is that outdated pages set the flag and then get deleted. Any process or thread that still has the deleted page open will thus see the flag and reload to the new page (which should be created with the same name, replacing the original).

Internal data structure​

The first section of the structure contains a static 36-byte header:

1 byte status bit fields (1 = ready, 2 = exit, 4 = reload)
1 byte field_offset (where the field description starts)
4 bytes record_count - number of records present
4 bytes record_size - bytes per record
4 bytes record_startpos - position of record where ring starts
8 bytes records_deleted - amount of records no longer present
4 bytes records_present - amount of records currently present
2 bytes record_offset
8 bytes record_endpos - total amount of records added

This is followed by the field description.

Thus, currently,field_offset is always set to 36. This field is encoded so that the header can change size in the future without breaking backwards and forwards compatibility.

The field description looks like this:

@field_offset: (record_offset - field_offset) bytes fields:

  5 bits field name len (< 32), 3 bits typelen (1-5)
  len bytes field name string (< 32 bytes)
  1 byte field type (0x01 = RelAccX, 0x1X = uint, 0x2X = int, 0x3X = string (text), 0x4X = string (binary))
    if typelen > 1: rest-of-type-len bytes max len
    else, for 0xYX:
      Y=1/2: X+1 bytes maxlen (1-16b)
      Y=3/4: (16 << X) bytes maxlen (16b-256kb)

Lastly, the actual data follows in this format:

@record_offset: (record_count * record_size) bytes, in same order as the fields:
  0x01: RelAccX record (allows for arbitrarily deep nesting)
  0x1X: X+1 bytes uint data
  0x2X: X+1 bytes int data
  0x3X: At most maxlen bytes of string data, zero terminated
  0x4X: maxlen bytes of binary data (formatting to be determined out of band)

All integers are encoded in host byte order (since no out-of-machine accesses happen), except 24 bit fields (which are network byte order), since generally no native 24-bit field accessor is available.

Rules when working with RelAccX structures​

The underlying storage for the data is assumed to be initialized to all zeroes before any kind of access starts. If the used storage does not natively already provide this guarantee, it must be taken care of through other means. (Shared Memory has been tested on all major platforms to indeed provide this guarantee).

When using the structure as a ring buffer (always adding new records at the end, removing old records from the beginning), care must be taken that records_present never goes over records_count. The structure is only safe under that condition.

Finally, the flags and the rules listed in their descriptions must always be followed, otherwise the safety properties of the structure will be lost.

Printing RelAccX shared memory object ("page") contents​

The command line utility MistUtilRAX accepts a page name as its only argument, and will then print the current contents of the given page in human-readable format. This is very helpful when debugging RelAccX-related code.