Version 25 (modified by yaghoubzadeh, 6 years ago) (diff)


ipaaca Development Tutorial

NOTE: This document is still preliminary. Suggestions and clarification requests are welcome.

Basic definitions

IUs and Messages

The basic unit of information transmitted (shared) in the ipaaca system is the Incremental Unit (IU), based on the "General, abstract model of incremental dialogue processing" by Schlangen et al. An IU is an object characterized by the following basic attributes:

  • uid - a globally unique identifier
  • category - a string representing the broad category of data, e.g. "asrresults" for transmitting the results of ASR
  • owner - the buffer name (see below) that initially produced this IU
  • committed - a flag that specifies whether the owner is committed to this IU and its contents (that it will remain valid and is final)
  • payload - the IU payload: a map of string->string, free to use by the application
  • links - a map of string->string, representing the links of the IU (see below)

IUs are persistent objects. That means that they can be modified at any time (unless specified as read-only by the user), and any changes of published IUs are automatically transmitted as updates to all relevant parties. They can also be modified from the remote side, unless specified otherwise.

The Message is a special case of an IU: it is a non-persistent read-only version of the IU. It can be used whenever you just want to send current information (akin to lightweight message-passing systems, hence the name), without the possibility of later modification. The benefit is that Messages are only present for the time of reception and do not occupy additional cumulative recources.


IUs are objects that exist in Buffers. A program can have any number of OutputBuffers and InputBuffers. When a new IU has been created, it has to be placed in an OutputBuffer. It is thereby published.

InputBuffers that components have initialized have a list of category interests, set by the user. Whenever an IU (or Message) of said categories is published or modified anywhere on the system, the corresponding InputBuffers will receive a notification of this, along with the updated IU contents.

Changes to IUs that are not marked as read-only can be effected at both ends of the pipeline: by simply writing to an IU present in a Buffer. When an IU has been written to from the remote side, the owner of that IU will also be able to receive a notification of this on the OutputBuffer where the IU was originally placed. IUs that have not yet been published can be written to at leisure without generating any events anywhere.

Links: networks of IUs

The IU model offers the possibility of representing (light-weight) dependency networks for IUs. To this effect, ipaaca offers the links member of an IU, a map str->str. The keys inside the hash represent the link type (name), while the values are comma-separated lists of IU UIDs which represent the target IUs of outgoing dependency links of the present IU. For example, and as a reference convention, the "grounded-in" links ("grin") of an IU represent IUs upon which the data of the current IU is based. ( e.g. links["grin"] = "uid-of-the-evidence-iu" )

Full minimal code examples with annotations

All examples are confirmed to work. To just copy-paste and test them, please ensure that your PYTHONPATH is set to include ipaaca. One possibility would be (for the default build system configuration) to just export PYTHONPATH=$HOME/repo/ipaaca/deps/python:$PYTHONPATH

Python: IU receiver example / InputBuffer

import ipaaca
import time

# First we define our IU event handler function
# The arguments are:
#       iu: the IU object
#       event_type: one of ADDED, UPDATED, RETRACTED (for IUs proper) or MESSAGE (for Message-type IUs)
#       own: whether the IU is an objects of this process or a remote object (always False in this example)
def my_first_iu_handler(iu, event_type, local):
	if event_type == 'ADDED':
		# you can also branch iu.category if you have more than one interest for the buffer
		print( u'Received a new IU of category: ' + iu.category )
		print( u'  The UID is: ' + iu.uid )
		print( u'  The payload is: ' + unicode(iu.payload) )
		# accessing fields
		old_value = iu.payload['someKey']
		# Advanced stuff:
		# the following lines are a remote write and will only succeed for IUs (not Messages)
			print('Now changing a field on the remote IU ...')
			iu.payload['someKey'] = 'updatedByMeJustNow'
		except IUUpdateFailedError, e:
			print('Failed, retry it or something!')
			# Note: this means the IU has been changed remotely in the mean time.
			# The user is responsible for retrying / ignoring / whatever.

# make a new InputBuffer (you can produce any number), with the category interests as a second argument
my_inbuffer = ipaaca.InputBuffer('MyArbitraryName', ['interestingCategory'])
# register an IU event handler (you can register more than one)

print('Doing nothing and waiting for IUs (or you to press Ctrl-C) ...')
while True:

Python: IU sender example / OutputBuffer, with remote update handler

import ipaaca
import time

# Define a handler.
# Note: since we attach this handler only to an OutputBuffer, below,
# we can expect to only receive remote updates (setting own == True)
# No one stops you from attaching a handler to several different buffers though.
def my_remote_update_handler(iu, event_type, own):
	print( u'Received an update from the remote side.' )
	print( u'  Current payload is now: ' + unicode(iu.payload) )

myCategory = 'interestingCategory'
print ( 'Will emit an IU of category '+ myCategory + 'and listen for 3 sec ...' )

my_outbuf = ipaaca.OutputBuffer('MyExampleSender')
iu = ipaaca.IU(myCategory)
iu.payload = { 'someKey':'initialValue',  'anotherKey':'12.345' }

# wait 3 sec for any remote changes

Python: minimal example to manually send BML (current ipaaca interface*)

(* will be replaced by a richer interface soon (in progress; only minimal changes for users))

import ipaaca
import time

class TextSender(object):
	def __init__(self):
		self.outbuf = ipaaca.OutputBuffer('MyExampleBmlSender')
		self.bml_id = 1
	def send_text(self, text):
		iu = ipaaca.IU('bml')
		iu.payload = { 'bml':'<bml xmlns="" id="bml' + str(self.bml_id) + '"><speech id="s1"><text>' + text + '</text></speech></bml>' }
		self.bml_id += 1

ts = TextSender()
ts.send_text('Hallo Welt.')
ts.send_text('Dies ist noch ein Satz.')



Reply here to give additional information or ask questions.