June 25, 2015
This document describes the protocol used by the 2015 RingBuffer library to stream ring buffer samples from the application whose buffer is being filled from a Usb interface (the server) to a client application.
This information is not needed if you use the RingBuffer library to receive the stream in Windows, Linux or OSX. This library will automatically uncompressed and unpacked the stream into a regular ring buffer just as it exists on the server. The test program "RingBuffer_SyncTest" included in this download package works uses this library and just as well as a client program as a server program. Its ring buffer is accessed in exactly the same way in either mode.
The information presented here is needed if you want to receive the stream on some other system, e.g. a tablet or a smartphone. The Java code below has been extracted from the module "RemoteAcquisitionFragment.java" in the sample Android App included in this download.
The server listens for connection requests on its "listen_port". This port defaults to 3113 but can be set to some other value in the server's parameter file called "ringbuffer.ini".
When a connection request is received, the server sends a 32 4-byte word (128 byte) message to the client that contains only two 4 byte binary numbers:
word[0] = 58 word[1] = 16296 word[2] = 0 ... word[31] = 0
The server then expects a 32 4-byte word (128 byte) reply from the client detailing the
channels wanted by the client. Channels are requested in 4-byte binary pairs that define
up to 16 channel ranges. Channels are numbered from 1. Channel 1 (the sync channel) and
channel 2 (the status channel) are always included and need not be explicitly requested.
Channels must be in ascending numeric order. The first word containing a 0 marks the end
of the ranges. For example, the client might reply:
word[0] = 1 word[1] = 58 word[2] = 0 ... word[31] = 0This requests that all available channels be sent, channel range 1-58.
The server searches its ring buffer for the start of channel set and begins the stream operation. This means that first channels transferred are always a sync word and a status word.
Before sending samples the server selects the requested channels into an intermediate buffer and compresses that buffer. Compression means dropping the low order zero byte in each channel word. The following simple compression scheme is used:s1a s1b s1c 0 s1a s1b s1c s4a s2a s2b s2c 0 becomes--> s2a s2b s3c s4b s3a s3b s3c 0 s3a s3b s3c s4c s4a s4b s4c 0
The simplest way for the client to deal with the Tcp stream is to work with 12 bytes at a time, 3 bytes per sample x 4 samples. When unpacked this will generate 4 4-byte sample words.
When uncompressing, unless there is only 4 channels being transferred, a full set of streamed channels (numChannelsXfr) will require uncompressing multiple sets of 4 sample words. Since the numChannelsXfr may not be a multiple of 4, there may be 1, 2 or 3 samples left over if unpacking for just one channel set. This overflow is avoided if channels are unpacked in groups of 4 channel sets at a time - numChannelsXfr*4 will always be a multiple of 4.
Uncompressing and unpacking should work with numChannelsXfr*4*3 streamed bytes at a time. This generates numChannelsXfr*4 sample words.
In this code:
// always uncompress and inflate a multiple of 4 sample sets
int numBytesPer4Sets = numChannelsXfr*4*3;
numNewBytesXfr = (numNewBytesXfr/numBytesPer4Sets)*numBytesPer4Sets;
int numNewSets = numNewBytesXfr/(numChannelsXfr*3);
int s[4]; // 4 output samples
s[0] = s[1] = s[2] = s[3] = 0; // to suppress compiler's uninitialized warning
unsigned int ps1,ps2,ps3; // 3 input packed samples
ps1 = ps2 = ps3 = 0; // to suppress compiler's uninitialized warning
int srcx = lastSrcx;
if (srcx == ints2UseXfr)
srcx = 0;
int destx = (nextDestx/4);
if (destx == ints2Use)
destx = 0;
// unpack 4th sample from the spare low order bytes in the 1st 3 samples
int sX = 4;
int chXfr = 0;
for (int numSets=0; numSets<numNewSets; numSets++)
{
for (int ch=0; ch<numChannels; ch++)
{
int sam = 0xffffff00; // not transferred
if (ch == channelXfrVector[chXfr])
{
if (++chXfr >= numChannelsXfr)
chXfr = 0;
if (sX > 3)
{
if (srcx+2 < ints2UseXfr)
{
ps1 = ((unsigned int *)samBufferXfr)[srcx++];
ps2 = ((unsigned int *)samBufferXfr)[srcx++];
ps3 = ((unsigned int *)samBufferXfr)[srcx++];
if (srcx == ints2UseXfr)
{
srcx = 0;
}
}
else if (srcx+1 < ints2UseXfr)
{
ps1 = ((unsigned int *)samBufferXfr)[srcx++];
ps2 = ((unsigned int *)samBufferXfr)[srcx];
ps3 = ((unsigned int *)samBufferXfr)[0];
srcx = 1;
}
else if (srcx < ints2UseXfr)
{
ps1 = ((unsigned int *)samBufferXfr)[srcx];
ps2 = ((unsigned int *)samBufferXfr)[0];
ps3 = ((unsigned int *)samBufferXfr)[1];
srcx = 2;
}
s[0] = (int)(ps1 & 0xffffff00);
s[1] = (int)(ps2 & 0xffffff00);
s[2] = (int)(ps3 & 0xffffff00);
s[3] = (int)(((ps1&0xff)<<24) | ((ps2&0xff)<<16) | ((ps3&0xff)<<8));
sX = 0;
}
sam = s[sX++];
}
((int *)samBuffer)[destx++] = sam;
if (destx == ints2Use)
{
destx = 0;
}
} // ch < numChannels
} // numSets < numNewSets
// always uncompress and inflate a multiple of 4 sample sets int numBytesPer4Sets = numChannelsXfr*4*3; int numNewSetBytesXfr = (numNewBytesXfr/numBytesPer4Sets)*numBytesPer4Sets; numNewBytesXfr -= numNewSetBytesXfr; int numNewSets = numNewSetBytesXfr/(numChannelsXfr*3); int[] s = new int[4]; // 4 output samples s[0] = s[1] = s[2] = s[3] = 0; // to suppress compiler's uninitialized warning int ps1,ps2,ps3; // 3 input packed samples ps1 = ps2 = ps3 = 0; // to suppress compiler's uninitialized warning int srcx = 0; int destx = seamIndex; if (destx == sampleBufferSizeInts) destx = 0; // unpack 4th sample from the spare low order bytes in the 1st 3 samples int sX = 4; for (int numSets=0; numSets<numNewSets; numSets++) { int ch = 0; for (ch=0; ch<numChannelsXfr; ch++) { int sam = 0; int chXfr = channelXfrVector[ch]; if (sX > 3) {// time to reload 3 words to create 4 samples (s[0]-s[3]) ps1 = ((int)(recvSampleBuffer[srcx++])&0xff)| (((int)(recvSampleBuffer[srcx++])&0xff)<<8)| (((int)(recvSampleBuffer[srcx++])&0xff)<<16)| ((int)(recvSampleBuffer[srcx++])<<24); ps2 = ((int)(recvSampleBuffer[srcx++])&0xff)| (((int)(recvSampleBuffer[srcx++])&0xff)<<8)| (((int)(recvSampleBuffer[srcx++])&0xff)<<16)| ((int)(recvSampleBuffer[srcx++])<<24); ps3 = ((int)(recvSampleBuffer[srcx++])&0xff)| (((int)(recvSampleBuffer[srcx++])&0xff)<<8)| (((int)(recvSampleBuffer[srcx++])&0xff)<<16)| ((int)(recvSampleBuffer[srcx++])<<24); s[0] = (int)(ps1 & 0xffffff00); s[1] = (int)(ps2 & 0xffffff00); s[2] = (int)(ps3 & 0xffffff00); s[3] = (int)(((ps1&0xff)<<24) | ((ps2&0xff)<<16) | ((ps3&0xff)<<8)); sX = 0; } sam = s[sX++]; numSamples++; } // ch < numChannels } // numSets < numNewSets
If the server detects "buffer overrun" (i.e. the client isn't taking the stream bytes fast enough), the server will close the connection. If the client re-connects within 5 seconds, streaming will resume (sending the same channels requested), on a channel set boundary (i.e. sync and status will be the first channels transferred). However because of the "overrun" condition some complete channel sets will have been missed by the client.
If the client closes the connect, the server considers the stream request finished. Any new connection must start with the 32 word message exchange described above to specify the set of channels wanted.