TODO

Topmost 3 tasks:
 ☐ For DHT: use sss negotiation to establish secure connection first, then use DHT packets within the channel to PING, STORE, FIND_NODE, and FIND_VALUE.
   - this means channel needs an API for direct sending of packets
   - therefore FIRST task is to complete and test kex_initiator/kex_responder working over UDP
      - need to be able to successfully carry out channel negotiation
 ☐ FIRST chat version may work over packets over channel, without streams even!
     - just need to send and ACK each packet one after another, retransmit if not acked in 500ms
 ✔ Extract sss::link and other low-level communication helpers into libuia. (uia::comm) @done (15-10-18 03:40)

Structured Secure Streams:
 ☐ Need to differentiate between substreams received as result of user actions
   and substreams received as means to deliver datagrams.
   Might not be that easy, but possibly streams that are already closed for writing by peer can be
   treated as datagrams, while still open streams ... or we can just send a flag for it?

   Problem with audio_service is that some unreliable data packets (?) are being sent as
   datagram_substreams and are accepted as control stream instead. This confuses the hell out
   of audio_service.

   It may be that in SST this was also the cause for multiple substreams just mixing up?

  ✔ we actually receive a datagram packet from the other side if it's a datagram, not a substream so simply stuff it into a separate datagram queue? @done (14-01-28 07:43)
    Done this for now, need to investigate setting up reliable or too big datagrams as substreams still, write a test.

 ☐ Going Online->Offline is not tracked properly, should update when failing to connect or receiving disconnect signal.
    Looks like presence service stops functioning once there are more services enabled. Looks like the last service always wins - bug in protocol? This is bug in Old SST, check if still applicable.

 ☐ Read up on Multipath TCP (@sa sst spec 4.3)

 ☐ Support obtaining readable descriptions and lists of supported services/protocols (spec 4.4 end)
   -- started in base_stream.cpp

Keying Negotiation:
 ☐ wrap common crypto operations under krypto - e.g. managing public/secret keys, seal/unseal etc
 ☐ http://ianix.com/pub/curve25519-deployment.html
 ☐ also need to negotiate dynamic MSS (spec 3.9)
 ☐ cover KeyInitiator and KeyResponder with unit tests (need to fake randomness bits somehow
    - by generating non-random fixtures - have pre-defined secret/public keys in fixtures)

 ☐ run audio streaming test in both the simulation environment (file-to-file using arbitrary loss percentage) and real VoiceService environment (file-to-hw output using local network connection).
   - hard atm due to source/sink driver design, look at pjsip ports/master-port idea for proper driver

UI:
 ☐ Switching UI to QtQuick and Qt5
   ☐ Make calling window work (send signals to start/stop call, update status)
 ☐ Draw UI sketches for contacts list,
   ☐ chat,
   ☐ audio/video call window,
   ☐ file transfers,
   ☐ folder sync,
   ☐ bandwidth utilisation/packet throughput graph window (like in Source).
 ☐ Handle empty settings_provider config gracefully!
 ☐ Maintain partial discoloration for online icon based on "stall count"
   e.g. second stall online icon is 2/3 colored, third stall icon is 1/3 colored, then offline (online-66%, online-33%)
   first stall is ignored as possibly temporary glitch
 ☐ how to perform search? an ESC-interface sounds intriguing
 ☐ Public/friends-only choices on profile items
 ☐ Profile picture
 ☐ Log window for log display in real-time.

Voice calls:
 ☐ Handle dissync - Ulric got packets about 3 minutes (!) later,
   hangup signal didn't go through before that - really need high-prio session control chan (above)
   - need a test for command side-channel testing while non-reliable packets are transmitted.
     ☐ spam a lot of datagrams, while still receiving send abort command, observe that no more than
       some % of packets is received by the time command goes through.
 ✔ Establish clock skew estimation and use it to drive jitterbuffer.
   ☐ (Adjust received packets time to local machine time, drop too old packets immediately.)
      See gstreamer rtp jitterbuffer for an algorithm to reestimate skew as the session runs.

Chats:
 ☐ Sync history between instances
 ☐ Draw possible chat synchronization graph ideas between Alice's and Bob's personal cluser devices.
 ☐ Chat documentation on https://github.com/berkus/mettanode/blob/develop/ui/doc/chat_and_messaging.md
 ☐ Group chat.
 ☐ Chat join urls like metta:chat?authblob
 ☐ Event bells/notifications: friend online, chat/talk started, etc.
 ☐ Chat happens in chat room.
 ☐ Rooms may be arbitrarily nested forming chat labyrinth. (@sa closedcircles)
 ☐ History features:
    Навигация по времени, возможности просматривать лог как лог (сплошную ленту), а не отдельные сообщения, в идеале возможность отмечать куски лога как важные. - Юдж
    еще экспорт и копипаста в читаемом виде. - Юдж

  UI: "Advertise my membership in [#osdev] room to [Nobody [v]"

Screencasting/screensharing:
 ☐ Screencasting client
   ☐ Capture screen and encode to video stream? libvpx or something like that. Run experiments.
    ☐ Or, subtract one screen from another and lossless-encode the difference.

Storage (grabber/file sync/versioning):
 ☐ Bandwidth indicator during downloads - see how win8 copy dialog looks like.
 ☐ Show file transfer progress on both sending and receiving sides. Also inside chat.
 ☐ Make ongoing downloads automatically persist across restarts
 ☐ Initiate file resync operation when change is detected.
     Should work pretty much like file transfer, just the file save locations are predefined.
     Different changes cause different synchronization commands to be sent.
     E.g. moving a file issues RENAME, while deleting it issues REMOVE and changing it issues UPDATE_SYNC.
     What to do with conflicting changes? One node sends rename, one remove and one modify?
     Something like full-fledged git might be needed. At the very least a key-value like store, that resolves modifications using full file path
     as just another attribute (change in FILE_PATH attribute might cause a rename, for example).

     Try git model approach - files are in merkle trees, trees are hashed and upon change
     a Change of tree is recorded with the link to previous (parent) tree(s).
     Files are like git blobs but assembled of multiple chunks. (See git implementation?)

☐ Synchronized objects - some blobs with associated metadata. Blob structure is not defined and is not interesting to the synchronization layer. Metadata is a key-value store, with fine-grained synchronization.
☐ Grab files from a specified directory a-la Dropbox.
☐ If metadata points to a blob not in the local store, system tries to find the blob on other devices and synchronise it.
☐ For the first version storage area can be limited to devices within the cluster. in general, blobs could be redundantly and securely stored on any nodes across the network.
☐ monitor Files folder for stuff to sync.
☐ metadata assignments - how? extract as much as possible automatically.
 ☐ synchronize arbitrary file chunks and their metadata between devices in the cluster (hardcoded for start).
 ☐ sync git-like file trees and chunk blobs
   ☐ @sa merkledag
   ☐ split a file into a number of variable-size blocks
     ☐ https://en.wikipedia.org/wiki/Rabin_fingerprint can be used to implement shift-resistant block splitting.
   ☐ record each block hash into a file record
   ☐ record metadata block(s), HMAC it/them - should it be fixed size or some more manageable chunks?
   ☐ add metadata block(s) hash(es) into a file record and then HMAC the entire record
   ☐ record each file record into a tree object and HMAC it
   ☐ go up the tree adding subtrees as records, just like git trees, HMAC each tree
   ☐ now since the tree has changed, record a Change, link it to parent Change, sign and HMAC it. - tag HEAD onto the latest change now
   ☐ this is where tree sync will start - other nodes compare their own HEAD changes with this node and perform a sync-fetch to align histories.
      in general case sync doesn't have to bring in all changes of the history, e.g. each Change object lineage may be synced, but for the file blocks
      you only need to know the actual contents of the latest version and where to find the earlier version in case user takes a peek.
 ☐ http://ailev.livejournal.com/1127145.html

Grabber prototype:
  ☐ Simple feature - grab YouTube "Favourites video" playlist, download all videos from there and store them.
      https://developers.google.com/youtube/v3/ - api documentation
      Needs OAuth to authenticate the user before grabbing his favourites.
        - We need certain auth creds store to keep things for our app.
           - Does Keychain allow storing this per-app? Check Apple docs.
           - What could be used on other systems?
  ☐ If user favourites is public, no need to auth?
  ☐ Do that periodically to fetch new videos.
  ☐ Use https://github.com/google/google-api-cpp-client/ - CRAP!
  ☐ There's `o2' qt library for oauth 1.0 and 2.0 - could use for login?
  ☐ This is against the TOS!

Personal Cluster Management (Naming):
 ☐ should be possible to add a device to personal device "cluster", thus allowing it to search the entire cluster and modify (optionally) data in the cluster.
 ☐ should be possible to remove a device from cluster, there might be no access to this device anymore (e.g. lost phone)
 ☐ devices in the cluster always keep track of status for other devices in the cluster and synchronize with them often.
 ☐ search between devices is performed in metadata index, which devices try to synchronise with priority to other objects.

Routing:
 ☐ Connect devices to each other, overlay routing lets device find another device and set up connection.
 ☐ LAN/Bonjour node discovery.
 ☐ Regserver deregistration on logout is added - should notify active connections somehow too?

 ☐ BUG: Crashes on access to client_profile.endpoints() if no endpoints was provided by the client.

 ☐ Possible intermediate step for work below:
   Handle standard ROU messages but from within link::receive()
 ☐ Regserver uses unencrypted network traffic.
   ☐ move regserver to a regular sss protocol
   ☐ run regserver inside host on the same port
   ☐ regserver shall be integral part of each participating node
   ☐ regserver will become a part of p2p directory when DHT is implemented
 ☐ Extract regserver logic/state management to remove code duplication between old and new regserver.

Most immediate usability necessities:
 ☐ Some documentation, both user and technical
   - some sss doc posted on atta-metta.net
 ☐ Automatic launching on host OS login

 ☐ write SSS spec as an RFC draft - complete unfinished SSS spec in form of RFC document
   ☐ assign IETF RFC number, see http://www.ietf.org/id-info/
 ☐ finish implementing wire protocol to specs.  Specifically:
   ☐ receiver-directed flow control
   ☐ datagram reassembly
   ☐ stream attachment and detachment
   ☐ proper stream ID allocation & assignment
   ☐ implement and use sequencing barriers properly (spec 3.4.1, also Figure 15 page 18)
 ☐ specify and fix negotiation protocol format

 ☐ give priority to local-network Endpoint addresses first (need to match network against available interfaces' IPs)
   ☐ match received endpoints against known local endpoint addresses (not external endpoint)
     ☐ longest matches get priority
     ☐ after all are matched, match on external endpoint address, this way EIDs sitting on the same
       external endpoint are supposed to connect directly through the NATs routing table.
 ✔ possible DOS if localhost is specified in Endpoints? yep, tested and true, fix. @done (13-11-04 01:14)
   ☐ first, never ever include loopback addresses in endpoints list
   ✔ second, if ever encountered, just ignore those @done (13-11-04 01:14)

  Possibly PID (Proportional-Integral-Differential) flow control.

sss/shell:
 ☐ unlock console fds once the shell connection ceases
 ☐ after some waiting time the streams disconnect and mix up
 ☐ Deploy nshd to motoko?

Testing:

 ☐ Set protocol specifications and make unit/func tests covering this spec.
   ☐ Working connect/disconnect/reconnect/etc
     ☐ See regress test in SST

 ☐ unit tests, unit tests, unit tests

substreams
 ☐ create a lot of substreams rapidly, hold onto them, use them once in a while
 ☐ create and destroy a lot of substreams in rapid succession
 ☐ rapidly create a deep hierarchy of nested substreams
   - see test_substreams

datagrams
 ☐ send a stream of datagrams of widely varying size
 ☐ send a mix of datagram substreams and real two-way substreams
 ☐ send datagrams with a control side-channel which manages the datagram sending, confirm
   that side-channel has priority.

flow control
 ☐ flow control for byte streams
 ☐ receive window borrowing
 ☐ receive window borrowing for datagram substreams
 ☐ inherited versus independent receive windows

NAT and UPnP:
 ☐ Integrate UPnP into sss::host. - should find upnp/natpnp and open necessary port from config
   when instantiating host, maintain it open for duration of host existence.
 ☐ Integrate hole punching machinery with sss::host.
 ☐ Assign a new `endpoint` with public IP and UPnP-assigned port number.
   ☐ Use it to register on regserver as primary.
     Right now regserver gives us this external IP address.
 ☐ UPnP event variables that need to be checked:
     NATEnabled - if not, portmapping setting will fail.
     ConnectionStatus - if not connected, setting forwarding is useless.
     PortMappingNumberOfEntries - if changed, might need to re-read the actual mapping information.
     PortMappingLeaseDuration - is decremented and needs to be checked regularly, some seconds before the lease time out the lease must be extended for our port forwardings.
     Therefore, we need to maintain the status of UPnPRouter endpoint.
 ☐ rfc5766-turn-server
 ☐ TRAM http://www.ietf.org/mail-archive/web/rtcweb/current/msg09823.html

Cross-compile:
  QtCreator xcompile to all platforms http://habrahabr.ru/post/198142/
  use mingw to cross-compile Qt for windows