wiki:BMLT

Version 2 (modified by anonymous, 8 years ago) (diff)

--

BML Twente

The BML Twente extension adds the specification of physical controllers, procedural animation, transition animations and synchronization to anticipated events to BML. Here we present a quick overview of the BMLT tags and their syntax. The namespace for BMLT is  http://hmi.ewi.utwente.nl/bmlt, we use bmlt as the prefix for that namespace throughout this document.

The BMLT behavior elements

<bmlt:parameter>

Used to pass parameter values to other BMLT elements, such as <bmlt:controller>, <bmlt:procanimation>, <bmlt:keyframe> or <bmlt:transition>.

AttributeTypeUseDescription
name  OpenSetItemrequiredparameter name
value string requiredparameter value

<bmlt:keyframe>

Describes a keyframe (or mocap) animation. Parameters can be passed using <bmlt:parameter>. The joints parameter selects the joints the animation should act upon. This is useful to subtract motion on a subsection of the body from full body mocap. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Mirror is always applied before joint selection.

AttributeTypeUseDescription
name OpenSetItemrequiredname of the keyframe animation

Example:

<bmlt:keyframe id="v1" name="plain_rhand">
  <bmlt:parameter name="joints" value="r_shoulder r_elbow r_wrist"/>
  <bmlt:parameter name="mirror" value="false"/>
</bmlt:keyframe>

<bmlt:audiofile>

Behavior that plays an audio file. Useful to, e.g., play a sound effect when the avatar claps his hands.

AttributeTypeUseDescription
filenamefilerequiredname of the file containing the audio

Example:

<bmlt:audiofile id="v1" filename="c:/audio/clap.wav" start="3"/>

<bmlt:controller>

Describes a physical controller behavior. Parameters can be passed to the controller using <bmlt:parameter>.

AttributeTypeUseDescription
class OpenSetItemrequiredJava class name of the controller

Example: enable the balance controller with pelvis height at 1.3m (resulting in a balance motion with unbend legs) and slightly increased stiffness.

<bmlt:controller id="balance1" class="BalanceController" start="0" end="10">
  <bmlt:parameter name="pelvisheight" value="1.3"/>
  <bmlt:parameter name="stiffnessmultiplier" value="1.1"/>
</bmlt:controller>

<bmlt:procanimation>

Describes a procedural animations. Parameters can be passed using <bmlt:parameter>. Like in keyframe animation, the joints parameter selects the joints the animation should act upon. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Again, mirror is always applied before joint selection.

AttributeTypeUseDescription
name OpenSetItemrequiredname of the procedural animation

Example: conduct a procedural 3 beat gesture with amplitude 5.

<bmlt:procanimation id="beat1" name="3-beat" start="3">
  <bmlt:parameter name="a" value="5"/>
</bmlt:procanimation>

<bmlt:transition>

Creates a transition animation. The start and end pose of the transition are determined automatically from its surrounding motions.

AttributeTypeUseDescription
class OpenSetItemrequiredJava class name of the transition used

Example: create a simple slerp transition on the arm's rotations, moving from the neutral pose toward the start arm pose of the conducting gesture

<bmlt:transition id="trans1" class="SlerpTransition" start="1" end="2">
  <bmlt:parameter name="joints" value="r_shoulder,r_elbow,r_wrist,l_shoulder,l_elbow,l_wrist"/>
</bmlt:transition> 
<bmlt:procanimation id="beat1" name="3-beat" start="trans1:end"/>

<bmlt:blinkmodelchange>

Dennis, is this still up-to-date?

Changes the parameters of the automatic blinking behavior. Waiting time between two automatic eye blinks is in the range [averagewaitingtime-range..averagewaitingtime+range]. Note: cannot have an end time (bmlt:blinkmodelchange is always persistent behavior). If automatic blink behavior was not turned on for the Virtual Human, bmlt:blinkmodelchange behaviors will be dropped.

AttributeTypeUseDescription
averagewaitingtimenonnegative floatrequiredaverage waiting time between blinks. '0' means to turn off automatic blinking behavior.
rangenonnegative floatrequiredmaximum variation in waiting time

Example:

<bmlt:blinkmodelchange id="bmc1" start="5" averagewaitingtime="0.5" range="0.2"/>

<bmlt:noise>

Generates a noise (typically Perlin noise) animation on a joint, specified by the type of noise, joint id and custom parameters.

AttributeTypeUseDescription
jointstringrequiredjoint id
type OpenSetItemrequiredtype of the noise

Example: <bmlt:noise id="noise1" type="perlin" joint="vl5" start="0" end="100">

<bmlt:parameter name="basefreqx" value="0.5"/> <bmlt:parameter name="baseamplitudex" value="0.05"/>

</bmlt:noise>

<bmlt:facemorph>

Dennis, is this still in use?

<bmlt:interrupt>

The BMLT interrupt behavior provides the capability of specifying precisely when specific running or scheduled behaviors should be interrupted.

AttributeTypeUseDescription
targetstringrequiredid of the target BML block to interrupt
includecomma separated list of stringsoptionalset of behaviors to interrupt in the target BML block (unspecified is all)
excludecomma separated list of stringsoptionalset of behaviors NOT to interrupt in the target BML block (unspecified is none)

Interrupt all behaviors in BML, except bml1:speech1 and bml1:gesture1.

<bmlt:interrupt id="i1" target="bml1" start="shake1:stroke" exclude="speech1,gesture1"/>

<bmlt:parametervaluechange>

The parametervaluechange behavior allows the modification of certain parameter values of ongoing behavior.

AttributeTypeUseDescription
targetstringrequiredid of the behavior the parametervaluechange targets
paramIdstringrequiredid of the targeted parameter

Change the volume of bml1:speech1:

<bmlt:parametervaluechange target="bml1:speech1" paramId="volume"
start="bml1:speech1:sync1" stroke="bml1:speech1:sync1+1">
<bmlt:trajectory type="linear" targetValue="90"/>
</bmlt:parametervaluechange>

The bmlt:parametervaluechange behavior contains a trajectory element that specifies the trajectory and value of the parameter that is changed.

AttributeTypeUseDescription
targetValuefloatrequiredtarget value of the parameter
typestringrequiredtype of the trajectory (e.g. linear, instant)
initialValuefloatoptionalstart value of the parameter, if omitted the start value is obtained from the targeted behavior

The parametervaluechange behavior sets the parameter value on the target behavior according to the trajectory, in such a way that the targetValue is reached at the stroke of the parametervaluechange behavior. It is not allowed to constrain the end sync of the parametervaluechange behavior; it ends automatically when the target value is achieved.

Preplanning and activation

Scheduling a BML block typically takes a non-negligible amount of time, especially if the timing of speech is to be obtained through speech synthesis software. This is problematic for developing highly responsive virtual humans. BMLT provides pre planning as a mechanism to construct a behavior plan that can be activated later on. In a typical usage scenario of pre-planning, the SAIBA Behavior Planner already knows what behavior to execute, and wants to execute it (near) instantly later on, for example in reaction to some event such as an incoming response from the user. Preplanning is set up for a BML block, using the BMLT preplan attribute in that block. Preplanned BML blocks can be activated using an BMLT activate behavior. The preplanned BML block is activated as soon the activate behavior starts. Preplan bml1:

<bml id="bml1" bmlt:preplan="true">
...
</bml>

<bmlt:activate>

The bmlt:activate behavior activates a preplanned block.

AttributeTypeUseDescription
targetstring|requiredid of the target block to activate

Example:

<bmlt:activate id="a1" target="bml1"/>

Synchronization to Predicted Events

We allow synchronization to external, possibly predicted events. This does not require an extension of BML, other than allowing a synchronization to reference ids that are not in the BML script. Example: link conducting beat 1, 2 and 3 of a procedural conducting to tick 1, 2 and 3 of a metronome.

<bmlt:procanimation id="conduct1" name="3-beat"/>
<constraint id="c1">
  <synchronize ref="conduct1:start">
    <sync ref="anticipators:metronome1:tick1"/>    	
  </synchronize>
  <synchronize ref="conduct1:beat2">
    <sync ref="anticipators:metronome1:tick2"/>    	
  </synchronize>
  <synchronize ref="conduct1:beat3">
    <sync ref="anticipators:metronome1:tick3"/>    	
  </synchronize>  
</constraint>

Persistent bmlt behaviors

Some BMLT behaviors are persistent (currently just bmlt:controller). They adhere to the same persistent patterns as gaze and posture in core BML. Example:
Specification of a persistent balance controller:

<bml id="bml1" xmlns:bmlt="http://hmi.ewi.utwente.nl/bmlt">
   <bmlt:controller id="balance1" class="BalanceController" start="1"/>
</bml>

Mutually exclusive behavior using replacement groups

Each BML behavior can be assigned a replacement group using a bmlt:parameter. Elckerlyc ensures that at most one behavior of each replacement group is played at the same time. The played behavior is the behavior with the most recent start time.

Example:

<bml id="bml1" xmlns:bmlt="http://hmi.ewi.utwente.nl/bmlt">
  <bmlt:controller id="balance1" class="BalanceController">
    <bmlt:parameter name="pelvisheight" value="1.2"/>
    <bmlt:parameter name="replacementgroup" value="balance"/>
  </bmlt:controller>
  <bmlt:controller id="balance1" class="BalanceController" start="4" end="8">
    <bmlt:parameter name="pelvisheight" value="0.8"/>
    <bmlt:parameter name="replacementgroup" value="balance"/>
  </bmlt:controller>
</bml>

Balances with stretched knees (pelvisheight = 1.2m) from time = 0 till time = 4, balances with bend knees (pelvisheight = 0.8m) from time = 4 till time = 8, then balances with stretched knees again.

BMLT behaviors can overwrite persistent core bml behaviors by using their name as replacement group:

<bml id="bml1" xmlns:bmlt="http://hmi.ewi.utwente.nl/bmlt">
  <posture id="pose1" stance="standing" shape="open" part="lower"/>	
  <bmlt:controller id="balance1" class="BalanceController" start="4">
    <bmlt:parameter name="pelvisheight" value="0.6"/>
    <bmlt:parameter name="replacementgroup" value="posture"/>
  </bmlt:controller>
</bml>

BMLT description extensions

Most BMLT behaviors may also be used as a description extension for a core BML behavior. For example, procanimation can be used as description extension for gesture and controller can be used as description extension for posture.

<gesture start="3" id="beat1" type="LEXICALIZED" lexeme="conduct-3-beat" hand="RIGHT">
  <description priority="1" type="bmlt:procanimation">
    <bmlt:procanimation id="beat1" name="conduct-3-beat" start="3">
      <bmlt:parameter name="a" value="5"/>
      <bmlt:parameter name="hand" value="RIGHT"/>
    </bmlt:procanimation>	
  </description>
</gesture>

BMLT also supports several speech description extensions, including SSML, Microsoft SAPI and various MaryTTS specifications.

Other Description Extensions Implemented by Elckerlyc

<speech> using Microsoft Speech API

Example:

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="1" type="application/msapi+xml">
      <sapi>I'm speaking <spell>BML</spell>.</sapi>
    </description>  	
  </speech>   
</bml>

<speech> using SSML

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="1" type="application/ssml+xml">
      <speak xmlns="http://www.w3.org/2001/10/synthesis">
      I'm <prosody pitch="high">speaking</prosody> BML.
      </speak>
    </description>  	
  </speech>   
</bml>

<speech> using Mary TTS

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="10" type="maryxml">
      <maryxml xmlns="http://mary.dfki.de/2002/MaryXML">
      I'm speaking BML.
      </maryxml>
    </description>  	
  </speech>   
</bml>

BMLT Feedback

In addition to the feedback specified in core BML, Elckerlyc provides global timestamps in all its feedback messages and additionally a local time stamp (time in seconds since start of the BML performance) for Sync-Point Progress Feedback. Elckerlyc also provides information on the planning status of a BML request, using Planning Start and Planning Finished feedback.

Scheduling Start Feedback

Notifies the behavior planner that scheduling of a requested BML performance has begun.

The message includes:

  • A character identifier
  • A BML block identifier
  • A global timestamp
  • The predicted start time of the BML block

Scheduling Finished Feedback

Notifies the behavior planner that scheduling of a requested BML block is finished.

The message includes:

  • A character identifier
  • A BML request identifier
  • A global timestamp
  • The predicted start time of the BML block

The BMLT BML attributes

append-after

append-after(X) scheduling attribute instructs the Realizer to execute the new BML block immediately after all behaviors from the prior BML blocks on list X have finished Example:

<bml id="bml4" scheduling="append-after(bml2,bml3)">
...
</bml>

allowexternalrefs

The allowexternalrefs attribute is used to indicate that a BML block may contain time constraints that refer to behaviors in other (external) BML blocks. Such references are of the form bmlid:behaviorid:syncid. Example:

<bml id="bml2" bmlt:allowexternalrefs="true">
<bmlt:parametervaluechange id="pvc1" target="bml1:speech1"
paramId="volume" start="bml1:speech1:sync1" stroke="bml1:speech1:end">
<bmlt:trajectory type="linear" initialValue="0" targetValue="100"/>
</bmlt:parametervaluechange>
</bml>

The interrupt shorthand

The interrupt attribute is a shorthand for the SAIBA Behavior Planner to remove a selected set of BML blocks from the multimodal behavior plan before scheduling the content of the BML block it is in. Example:

<bml id="bmlNew" bmlt:interrupt="bml1,bml2,..,bmln">
bmlNew content
</bml>

This example is a shorthand for:

  1. Send a BML block to the Realizer that interrupts bml1..bmln:
    <bml id="bmlInterrupt">
      <bmlt:interrupt id="interrupt1" target="bml1"/>
      <bmlt:interrupt id="interrupt2" target="bml2"/>
      ..
      <bmlt:interrupt id="interruptn" target="bmln"/>
    </bml>
    
  2. Wait for block end feedback of bmlInterrupt (to make sure bml1..bmln are properly removed from

the multimodal behavior plan).

  1. Send the new BML block bmlNew:
    <bml id="bmlNew">
    bmlNew content
    </bml>
    

The onStart shorthand

The onStart attribute is a shorthand for the SAIBA Behavior Planner to activate a selected set of BML blocks in the multimodal behavior plan whenever a certain BML block starts. Example:

<bml id="bmlNew" bmlt:onStart="bml1,bml2,..,bmln">
  bmlNew content
</bml>

This is a shorthand for:

<bml id="bmlActivate">
  <bmlt:activate id="activate1" target="bml1"/>
  <bmlt:activate id="activate2" target="bml2"/>
  ..
  <bmlt:activate id="activaten" target="bmln"/>
  bmlNew content
</bml>