wiki:BMLT

Version 16 (modified by welberge, 7 years ago) (diff)

--

BML Twente

The BML Twente extension adds the specification of physical controllers, procedural animation, transition animations and synchronization to anticipated events to BML. Here we present a quick overview of the BMLT tags and their syntax. The namespace for BMLT is http://hmi.ewi.utwente.nl/bmlt, we use bmlt as the prefix for that namespace throughout this document.

BMLT attribute extensions to core BML elements

<speech>

The voice attribute can be used to select another voice, e.g. to allow an agent to speak multiple languages.

<speech id="s1" bmlt:voice="myvoice"><text>Hello world.</text></speech>

After speech s1 is finished, the voice is restored to whatever was the default voice before.

<gaze>

The dynamic attribute can be used to indicate that the gaze target may be moving and should be tracked. By default, AsapRealizer's AnimationEngine assumes that the target is non-moving; gaze is directed towards the target position at the start of the gaze motion. This option is provided to users because AsapRealizer has a nicer looking movement model for static gaze.

<gaze bmlt:dynamic="true" id="gaze1" influence="NECK" target="greensphere"/>

The BMLT behavior elements

<bmlt:parameter>

Used to pass parameter values to other BMLT elements, such as <bmlt:controller>, <bmlt:procanimation>, <bmlt:keyframe> or <bmlt:transition>.

AttributeTypeUseDescription
name  OpenSetItemrequiredparameter name
value string requiredparameter value

<bmlt:keyframe>

Describes a keyframe (or mocap) animation. Parameters can be passed using <bmlt:parameter>. The joints parameter selects the joints the animation should act upon. This is useful to subtract motion on a subsection of the body from full body mocap. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Mirror is always applied before joint selection.

AttributeTypeUseDescription
name OpenSetItemrequiredname of the keyframe animation

Example:

<bmlt:keyframe id="v1" name="plain_rhand">
  <bmlt:parameter name="joints" value="r_shoulder r_elbow r_wrist"/>
  <bmlt:parameter name="mirror" value="false"/>
</bmlt:keyframe>

<bmlt:audiofile>

Behavior that plays an audio file. Useful to, e.g., play a sound effect when the avatar claps his hands.

AttributeTypeUseDescription
filenamefilerequiredname of the file containing the audio

Example:

<bmlt:audiofile id="v1" filename="c:/audio/clap.wav" start="3"/>

<bmlt:controller>

Describes a physical controller behavior. Parameters can be passed to the controller using <bmlt:parameter>.

AttributeTypeUseDescription
class OpenSetItemrequiredJava class name of the controller

Example: enable the balance controller with pelvis height at 1.3m (resulting in a balance motion with unbend legs) and slightly increased stiffness.

<bmlt:controller id="balance1" class="BalanceController" start="0" end="10">
  <bmlt:parameter name="pelvisheight" value="1.3"/>
  <bmlt:parameter name="stiffnessmultiplier" value="1.1"/>
</bmlt:controller>

<bmlt:procanimation>

Describes a procedural animations. Parameters can be passed using <bmlt:parameter>. Like in keyframe animation, the joints parameter selects the joints the animation should act upon. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Again, mirror is always applied before joint selection.

AttributeTypeUseDescription
name OpenSetItemrequiredname of the procedural animation

Example: conduct a procedural 3 beat gesture with amplitude 5.

<bmlt:procanimation id="beat1" name="3-beat" start="3">
  <bmlt:parameter name="a" value="5"/>
</bmlt:procanimation>

<bmlt:transition>

Creates a transition animation. The start and end pose of the transition are determined automatically from its surrounding motions.

AttributeTypeUseDescription
class OpenSetItemrequiredJava class name of the transition used

Example: create a simple slerp transition on the arm's rotations, moving from the neutral pose toward the start arm pose of the conducting gesture

<bmlt:transition id="trans1" class="SlerpTransition" start="1" end="2">
  <bmlt:parameter name="joints" value="r_shoulder,r_elbow,r_wrist,l_shoulder,l_elbow,l_wrist"/>
</bmlt:transition> 
<bmlt:procanimation id="beat1" name="3-beat" start="trans1:end"/>

<bmlt:blinkemitter>

Sets automatic blinking behavior with specified parameters. Waiting time between two automatic eye blinks is in the range [avgwaitingtime-range..avgwaitingtime+range]. Note: cannot have an end time (bmlt:blinkemitter is always persistent behavior). If automatic blink behavior was not turned on for the Virtual Human, bmlt:blinkemitter behaviors will be dropped. See ElckerlycDocumentation? under "Develop / VirtualHumanSpecs" for details on how to turn on automatic blink behavior. To change the parameters of the blinking behavior, you can either specify a new bmlt:blinkemitter behavior, which will override the previous one, or you can use a bmlt:parametervaluechange behavior (see example).

AttributeTypeUseDescription
avgwaitingtimenonnegative floatrequiredaverage waiting time between blinks. '0' means to turn off automatic blinking behavior.
rangenonnegative floatrequiredmaximum variation in waiting time

Example:

  <bmlt:blinkemitter id="blinkemitter1" start="0" range="0.5" avgwaitingtime="10"/>
  <bmlt:parametervaluechange id="pvc1" target="bml1:blinkemitter1" paramId="avgwaitingtime" start="20"> 
    <bmlt:trajectory type="instant" targetValue="1"/>
  </bmlt:parametervaluechange>

<bmlt:breathingemitter>

Sets automatic breathing behavior with specified parameters, animated by moving the shoulders slightly. Waiting time between two breaths is in the range [avgwaitingtime-range..avgwaitingtime+range]. Note: cannot have an end time (bmlt:breathingemitter is always persistent behavior). If automatic breathing was not turned on for the Virtual Human, bmlt:breathingemitter behaviors will be dropped. See ElckerlycDocumentation? under "Develop / VirtualHumanSpecs" for details on how to turn on automatic breathing. To change the parameters of the breathing behavior, you can either specify a new bmlt:breathingemitter behavior, which will override the previous one, or you can use a bmlt:parametervaluechange behavior (see example).

AttributeTypeUseDescription
avgwaitingtimenonnegative floatrequiredaverage waiting time between breaths. '0' means to turn off automatic blinking behavior.
rangenonnegative floatrequiredmaximum variation in waiting time

Example:

  <bmlt:breathingemitter id="breathingemitter1" start="0" range="2" avgwaitingtime="10"/>
  <bmlt:parametervaluechange id="pvc1" target="bml1:breathingemitter1" paramId="avgwaitingtime" start="20"> 
    <bmlt:trajectory type="instant" targetValue="1"/>
  </bmlt:parametervaluechange>

<bmlt:noise>

Generates a noise (typically Perlin noise) animation on a joint, specified by the type of noise, joint id and custom parameters.

AttributeTypeUseDescription
jointstringrequiredjoint id
type OpenSetItemrequiredtype of the noise

Example:

<bmlt:noise id="noise1" type="perlin" joint="vl5" start="0" end="100">
  <bmlt:parameter name="basefreqx" value="0.5"/>
  <bmlt:parameter name="baseamplitudex" value="0.05"/>
</bmlt:noise>

Note that Perlin noise behaviors have the following possible bmlt parameters: basefreqx, basefreqy, basefreqz, baseamplitudex, baseamplitudey, baseamplitudez, persistencex, persistencey, persistencez

<bmlt:facemorph>

Directly controls a face morph target defined on the avatar mesh.

AttributeTypeUseDescription
targetnamestringrequiredname of morph target
intensityfloatoptionalamnount to which the morph target is activated

Example:

<facemorph id="face1" targetname="Body_NG-mesh-morpher-Smile01-9" start="0" end="5"/>

<bmlt:interrupt>

The BMLT interrupt behavior provides the capability of specifying precisely when specific running or scheduled behaviors should be interrupted.

AttributeTypeUseDescription
targetstringrequiredid of the target BML block to interrupt
includecomma separated list of stringsoptionalset of behaviors to interrupt in the target BML block (unspecified is all)
excludecomma separated list of stringsoptionalset of behaviors NOT to interrupt in the target BML block (unspecified is none)

Interrupt all behaviors in BML, except bml1:speech1 and bml1:gesture1.

<bmlt:interrupt id="i1" target="bml1" start="shake1:stroke" exclude="speech1,gesture1"/>

<bmlt:parametervaluechange>

The parametervaluechange behavior allows the modification of certain parameter values of ongoing behavior.

AttributeTypeUseDescription
targetstringrequiredid of the behavior the parametervaluechange targets
paramIdstringrequiredid of the targeted parameter

Change the volume of bml1:speech1:

<bmlt:parametervaluechange target="bml1:speech1" paramId="volume"
start="bml1:speech1:sync1" stroke="bml1:speech1:sync1+1">
<bmlt:trajectory type="linear" targetValue="90"/>
</bmlt:parametervaluechange>

The bmlt:parametervaluechange behavior contains a trajectory element that specifies the trajectory and value of the parameter that is changed.

AttributeTypeUseDescription
targetValuefloatrequiredtarget value of the parameter
typestringrequiredtype of the trajectory (e.g. linear, instant)
initialValuefloatoptionalstart value of the parameter, if omitted the start value is obtained from the targeted behavior

The parametervaluechange behavior sets the parameter value on the target behavior according to the trajectory, in such a way that the targetValue is reached at the stroke of the parametervaluechange behavior. It is not allowed to constrain the end sync of the parametervaluechange behavior; it ends automatically when the target value is achieved.

Preplanning and activation

Scheduling a BML block typically takes a non-negligible amount of time, especially if the timing of speech is to be obtained through speech synthesis software. This is problematic for developing highly responsive virtual humans. BMLT provides pre planning as a mechanism to construct a behavior plan that can be activated later on. In a typical usage scenario of pre-planning, the SAIBA Behavior Planner already knows what behavior to execute, and wants to execute it (near) instantly later on, for example in reaction to some event such as an incoming response from the user. Preplanning is set up for a BML block, using the BMLT preplan attribute in that block. Preplanned BML blocks can be activated using an BMLT activate behavior. The preplanned BML block is activated as soon the activate behavior starts. Preplan bml1:

<bml id="bml1" bmlt:preplan="true">
...
</bml>

<bmlt:activate>

The bmlt:activate behavior activates a preplanned block.

AttributeTypeUseDescription
targetstring|requiredid of the target block to activate

Example:

<bmlt:activate id="a1" target="bml1"/>

Synchronization to Predicted Events

We allow synchronization to external, possibly predicted events. This does not require an extension of BML, other than allowing a synchronization to reference ids that are not in the BML script. Example: link conducting beat 1, 2 and 3 of a procedural conducting to tick 1, 2 and 3 of a metronome.

<bmlt:procanimation id="conduct1" name="3-beat"/>
<constraint id="c1">
  <synchronize ref="conduct1:start">
    <sync ref="anticipators:metronome1:tick1"/>    	
  </synchronize>
  <synchronize ref="conduct1:beat2">
    <sync ref="anticipators:metronome1:tick2"/>    	
  </synchronize>
  <synchronize ref="conduct1:beat3">
    <sync ref="anticipators:metronome1:tick3"/>    	
  </synchronize>  
</constraint>

BMLT description extensions

Most BMLT behaviors may also be used as a description extension for a core BML behavior. For example, procanimation can be used as description extension for gesture and controller can be used as description extension for posture.

<gesture start="3" id="beat1" type="LEXICALIZED" lexeme="conduct-3-beat" hand="RIGHT">
  <description priority="1" type="procanimation">
    <bmlt:procanimation id="beat1" name="conduct-3-beat" start="3">
      <bmlt:parameter name="a" value="5"/>
      <bmlt:parameter name="hand" value="RIGHT"/>
    </bmlt:procanimation>	
  </description>
</gesture>

BMLT also supports several speech description extensions, including SSML, Microsoft SAPI and various MaryTTS specifications.

Other Description Extensions Implemented by AsapRealizer

<speech> using Microsoft Speech API

Example:

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="1" type="application/msapi+xml">
      <sapi>I'm speaking <spell>BML</spell>.</sapi>
    </description>  	
  </speech>   
</bml>

<speech> using SSML

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="1" type="application/ssml+xml">
      <speak xmlns="http://www.w3.org/2001/10/synthesis">
      I'm <prosody pitch="high">speaking</prosody> BML.
      </speak>
    </description>  	
  </speech>   
</bml>

<speech> using Mary TTS

Find more information: UsingMaryTTS

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="10" type="maryxml">
      <maryxml xmlns="http://mary.dfki.de/2002/MaryXML">
      I'm speaking BML.
      </maryxml>
    </description>  	
  </speech>   
</bml>

The BMLT BML attributes

The interrupt shorthand

The interrupt attribute is a shorthand for the SAIBA Behavior Planner to remove a selected set of BML blocks from the multimodal behavior plan before scheduling the content of the BML block it is in. Example:

<bml id="bmlNew" bmlt:interrupt="bml1,bml2,..,bmln">
bmlNew content
</bml>

This example is a shorthand for:

  1. Send a BML block to the Realizer that interrupts bml1..bmln:
    <bml id="bmlInterrupt">
      <bmlt:interrupt id="interrupt1" target="bml1"/>
      <bmlt:interrupt id="interrupt2" target="bml2"/>
      ..
      <bmlt:interrupt id="interruptn" target="bmln"/>
    </bml>
    
  2. Wait for block end feedback of bmlInterrupt (to make sure bml1..bmln are properly removed from

the multimodal behavior plan).

  1. Send the new BML block bmlNew:
    <bml id="bmlNew">
    bmlNew content
    </bml>
    

The onStart shorthand

The onStart attribute is a shorthand for the SAIBA Behavior Planner to activate a selected set of BML blocks in the multimodal behavior plan whenever a certain BML block starts. Example:

<bml id="bmlNew" bmlt:onStart="bml1,bml2,..,bmln">
  bmlNew content
</bml>

This is a shorthand for:

<bml id="bmlActivate">
  <bmlt:activate id="activate1" target="bml1"/>
  <bmlt:activate id="activate2" target="bml2"/>
  ..
  <bmlt:activate id="activaten" target="bmln"/>
  bmlNew content
</bml>

Discussion