README

PySPEAD

This is a fast and light Python implementation of the Streaming Protocol for the Exchange of Astronomical Data

New in 0.5.1:
    - Sanitised variable length string handling so as not to return array containing single string

New in 0.5:
    - Added experimental string handling using mkfmt(('s',8))

ToDo:
    - Unpacking is now pretty quick, but for arrays, could we get one more speed-up by unpacking data directly into
      a numpy array, rather than to a tuple? 

    - Right now, packets are pulled straight from the socket and put into a packet ringbuffer.  The packet
      ringbuffer then feeds a callback that consumes the packets and creates a heap.  While heap creation from
      packets is performed in C, a Python callback is currently used to hand the packets to SpeadHeap.
      (a) We could make heap assembly happen automatically, so that the Python callback is fed heaps directly.
      (b) While implementing this, we should also incorportate packet sorting by heap_cnt so that packets can
          arrive out of order, but heaps are assembled in order to the extent allowed by the available buffer.

    - Confusingly, there are currently two kinds of "heap" in spead.py: one is a SpeadHeap that is generated
      by a receiver, and one is a dictionary that is used by a transmitter.  We might move the heap => packets
      pipeline into C as well, and make all heaps of the SpeadHeap type.

    - Packet sorting is done as each packet is inserted, but it could be
      done faster by buffering them and then quicksorting

    - _spead.pack() is currently the limiting factor on transmit.  Can it be sped up?

    - tests/test_spead.py uses lots of explicit constants.  We should migrate to using the named constants
      provided in the module.
       
Known Issues: 
    - Occasionally MacOSX hangs forever in unittest test_spead.py TestTransportUDPrx.test_get_packets_term.  This
      seems to be b/c no data is being received.  Hanging seems to be more frequent when buffer_socket.cpp
      has the DEBUG flag set.  Linux doesn't seem to have the same problem.  Cause unknown.