wiki:LiveMocap

LiveMocap

Extensions to embed live motion capture performance of specific bodyparts in the realization. The namespace for live mocap is  http://asap-project.org/livemocap. Currently it supports the live insertion of FACS based facial expressions, of head movement and (experimental) of gaze. TODO: document gaze to a point, it seems only NAO support is available for that.

LiveMocapEngine setup

The LiveMocapEngine requires a set of input sensors, that are linked to AsapRealizer's output embodiments (also listed in the specification) through BML. See below for an example configuration of the LiveMocapEngine.

Example (see also the asaparmandia_livemocap.xml config in AsapRealizerDemo):

<Loader id="keyboardinput" loader="asap.livemocapengine.inputs.loader.KeyboardInputLoader"/>
<Loader id="graphicalembodiment" loader="hmi.renderenvironment.HmiRenderEmbodimentLoader">
...
</Loader>
<Loader id="livemocapengine" loader="asap.livemocapengine.loader.LiveMocapEngineLoader" requiredloaders="keyboardinput,graphicalembodiment">
  <input name="keyboardinput" interface="asap.livemocapengine.inputs.EulerInput"/>
  <input name="keyboardinput" interface="asap.livemocapengine.inputs.FACSFaceInput"/>
  <output name="graphicalembodiment" interface="hmi.headandgazeembodiments.FACSFaceEmbodiment"/>
  <output name="graphicalembodiment" interface="hmi.headandgazeembodiments.EulerHeadEmbodiment"/>
</Loader>

The live mocap behaviour elements

Each livemocap behavior has a sensor input, which links directly to an embodiment output. Both the sensor and the embodiment may be loaded through the AsapVirtualHuman XML (see above).

Example:

<bml xmlns="http://www.bml-initiative.org/bml/bml-1.0" id="bml1">
  <remoteHead xmlns="http://asap-project.org/livemocap" id="rhead1" input="keyboardinput" output="graphicalembodiment" start="0" end="10000"/>
  <remoteFaceFACS xmlns="http://asap-project.org/livemocap" id="rfacs1" input="keyboardinput" output="graphicalembodiment" start="0" end="10000"/>
</bml>

<livemocap:remoteHead>

AttributeTypeUseDescription
input string requiredinput sensor id
outputstring requiredoutput embodiment id

<livemocap:remoteFaceFACS>

AttributeTypeUseDescription
input string requiredinput sensor id
outputstring requiredoutput embodiment id

<livemocap:remoteGaze>

AttributeTypeUseDescription
input string requiredinput sensor id
outputstring requiredoutput embodiment id