wiki:IpaacaTutorial

Version 7 (modified by yaghoubzadeh, 6 years ago) (diff)

--

ipaaca Development Tutorial

INCOMPLETE - in progress

Basic definitions

IUs and Messages

The basic unit of information transmitted (shared) in the ipaaca system is the Incremental Unit (IU). An IU is an object characterized by the following basic attributes:

  • uid - a globally unique identifier
  • category - a string representing the broad category of data, e.g. "asrresults" for transmitting the results of ASR
  • owner - the buffer name (see below) that initially produced this IU
  • committed - a flag that specifies whether the owner is committed to this IU and its contents (that it will remain valid and is final)
  • payload - the IU payload: a hash of string->string, free to use by the application
  • links - a hash of string->string, representing the links of the IU (see below)

IUs are persistent objects. That means that they can be modified at any time (unless specified as read-only by the user), and any changes of published IUs are automatically transmitted as updates to all relevant parties. They can also be modified from the remote side, unless specified otherwise.

The Message is a special case of an IU: it is a non-persistent read-only version of the IU. It can be used whenever you just want to send current information (akin to lightweight message-passing systems, hence the name), without the possibility of later modification. The benefit is that Messages are only present for the time of reception and do not occupy additional cumulative recources.

Buffers

IUs are objects that exist in Buffers. A program can have any number of OutputBuffers and InputBuffers. When a new IU has been created, it has to be placed in an OutputBuffer. It is thereby published.

InputBuffers that components have initialized have a list of category interests, set by the user. Whenever an IU (or Message) of said categories is published or modified anywhere on the system, the corresponding InputBuffers will receive a notification of this, along with the updated IU contents.

Changes to IUs that are not marked as read-only can be effected at both ends of the pipeline: by simply writing to an IU present in a Buffer. When an IU has been written to from the remote side, the owner of that IU will also be able to receive a notification of this on the OutputBuffer where the IU was originally placed. IUs that have not yet been published can be written to at leisure without generating any events anywhere.

Full minimal code examples with annotations

This is a grey line, please ignore it :)

Python: IU receiver example / InputBuffer

import ipaaca
import time

# First we define our IU event handler function
# The arguments are:
#       iu: the IU object
#       event_type: one of ADDED, UPDATED, RETRACTED (for IUs proper) or MESSAGE (for Message-type IUs)
#       own: whether the IU is an objects of this process or a remote object (always False in this example)
def my_first_iu_handler(iu, event_type, local):
	if event_type == 'ADDED':
		# you can also branch iu.category if you have more than one interest for the buffer
		print( u'Received a new IU of category: ' + iu.category )
		print( u'  The UID is: ' + iu.uid )
		print( u'  The payload is: ' + unicode(iu.payload) )
		# accessing fields
		old_value = iu.payload['someKey']
		#
		# Advanced stuff:
		# the following lines are a remote write and will only succeed for IUs (not Messages)
		try:
			iu.payload['someKey'] = 'updatedByMeJustNow'
		except IUUpdateFailedError, e:
			print('Failed, retry it or something!')
			# Note: this means the IU has been changed remotely in the mean time.
			# The user is responsible for retrying / ignoring / whatever.

# make a new InputBuffer (you can produce any number), with the category interests as a second argument
my_inbuffer = ipaaca.InputBuffer('MyArbitraryName', ['interestingCategory'])
# register an IU event handler (you can register more than one)
my_inbuffer.register_handler(my_first_iu_handler)

print('Doing nothing and waiting for IUs (or you to press Ctrl-C) ...')
while True:
	time.sleep(1)