wiki:BMLT

BML Twente

The BML Twente extension adds the specification of physical controllers, procedural animation, transition animations and synchronization to anticipated events to BML. Here we present a quick overview of the BMLT tags and their syntax. The namespace for BMLT is http://hmi.ewi.utwente.nl/bmlt, we use bmlt as the prefix for that namespace throughout this document.

BMLT attribute extensions to core BML elements

BMLT offers a few attribute extensions that can be used to enhance the specification of core BML elements.

<speech>

The voice attribute can be used to select another voice, e.g. to allow an agent to speak multiple languages.

<speech id="s1" bmlt:voice="myvoice"><text>Hello world.</text></speech>

After speech s1 is finished, the voice is restored to whatever was the default voice before.

<gaze>

The dynamic attribute can be used to indicate that the gaze target may be moving and should be tracked. By default, AsapRealizer's AnimationEngine assumes that the target is non-moving; gaze is directed towards the target position at the start of the gaze motion. This option is provided to users because AsapRealizer has a nicer looking movement model for static gaze.

<gaze bmlt:dynamic="true" id="gaze1" influence="NECK" target="greensphere"/>

The BMLT behavior elements

AttributeTypeUseDescription
name  OpenSetItemrequiredparameter name
value string requiredparameter value

<bmlt:parameter>

Used to pass parameter values to other BMLT elements, such as <bmlt:controller>, <bmlt:procanimation>, <bmlt:keyframe> or <bmlt:transition>.

AttributeTypeUseDescription
name  OpenSetItemrequiredparameter name
value string requiredparameter value

<bmlt:facekeyframe>

Describes a keyframe (or mocap) animation of the face.

AttributeTypeUseDescription
name OpenSetItemoptionalname of the face keyframe animation
typeMORPH, FACS or FAPSrequiredMorph, action unit or MPEG4 animation

Example:

<bmlt:facekeyframe type="FAPS" id="fk1">
  <FaceInterpolator parts="36 35 34 33 32 31 38 37">
    0.00474 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
    0.01307 0.04468918 -0.06225586 -0.2604103 -1.616621 -0.27528763 -0.38114548 0.8409977 -0.522995
    0.0214 0.29462814 -0.33016205 0.21351814 -0.94491005 -0.020256042 -0.39058685 2.6735067 -1.1125088
    ...
  </FaceInterpolator>
</bmlt:facekeyframe>

<bmlt:keyframe>

Describes a keyframe (or mocap) animation. Parameters can be passed using <bmlt:parameter>. The joints parameter selects the joints the animation should act upon. This is useful to subtract motion on a subsection of the body from full body mocap. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Mirror is always applied before joint selection.

AttributeTypeUseDescription
name OpenSetItemoptionalname of the keyframe animation

Example playing back the plain_rhand animation (as defined in the gesturebinding):

<bmlt:keyframe id="v1" name="plain_rhand">
  <bmlt:parameter name="joints" value="r_shoulder r_elbow r_wrist"/>
  <bmlt:parameter name="mirror" value="false"/>
</bmlt:keyframe>

Rather than using the gesturebinding, <bmlt:keyframe> may also be used to specify keyframe animations directly using SkeletonInterpolator XML. E.g.:

<bmlt:keyframe id="kf1">   
    <SkeletonInterpolator xmlns="" rotationEncoding="axisangles" parts="skullbase" encoding="R">
 		0 0 1 0 0
		1 0 1 0 1
    </SkeletonInterpolator>
</bmlt:keyframe>

This is especially useful for the reproduction of motion in a corpus.

<bmlt:text>

Behavior that presents some text. Useful as to steer GUI information, or to subtitle speech. Example:

<bmlt:text id="t1">
Hello World.
</bmlt:text>

By default the text will scroll in at a speech rate. You can make use of <sync>'s (as in the speech element) inside the text to adjust the timing.

Example with subtitling:

<bml xmlns="http://www.bml-initiative.org/bml/bml-1.0"  id="bml1" xmlns:bmlt="http://hmi.ewi.utwente.nl/bmlt">
<speech id="s1">
<text>Hallo welt. Ich bin Billie.</text>
</speech>
<bmlt:text id="t1" start="s1:start" end="s1:end">
Hello world. I'm Billy.
</bmlt:text>
</bml>

<bmlt:audiofile>

Behavior that plays an audio file. Useful to, e.g., play a sound effect when the avatar claps his hands.

AttributeTypeUseDescription
fileNamefilerequiredname of the file containing the audio


Sync attributedescription
start
end

The fileName can be an URL (prefix with file:/ for files on your local system) or a file on the classpath (e.g. in the resource dir AsapRealizerDemo). Example:

<bmlt:audiofile id="v1" fileName="file:/c:/audio/clap.wav" start="3"/>

<bmlt:controller>

Describes a physical controller behavior. Parameters can be passed to the controller using <bmlt:parameter>.

AttributeTypeUseDescription
class OpenSetItemrequiredJava class name of the controller

Example: enable the balance controller with pelvis height at 1.3m (resulting in a balance motion with unbend legs) and slightly increased stiffness.

<bmlt:controller id="balance1" class="BalanceController" start="0" end="10">
  <bmlt:parameter name="pelvisheight" value="1.3"/>
  <bmlt:parameter name="stiffnessmultiplier" value="1.1"/>
</bmlt:controller>

<bmlt:procanimation>

Describes a procedural animations. Parameters can be passed using <bmlt:parameter>. Like in keyframe animation, the joints parameter selects the joints the animation should act upon. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Again, mirror is always applied before joint selection.

AttributeTypeUseDescription
name OpenSetItemrequiredname of the procedural animation

Example: conduct a procedural 3 beat gesture with amplitude 5.

<bmlt:procanimation id="beat1" name="3-beat" start="3">
  <bmlt:parameter name="a" value="5"/>
</bmlt:procanimation>

<bmlt:procanimationgesture>

Describes a procedural animation inline. Includes automatically generated preparation and retraction phases. Like in keyframe animation, the joints parameter selects the joints the animation should act upon. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Again, mirror is always applied before joint selection.

ProcAnimations may be specified within the procanimationgesture:

<bmlt:procanimationgesture id="ani">
   <ProcAnimation prefDuration="1" minDuration="1" maxDuration="1">
    <SkeletonInterpolator rotationEncoding="axisangles" parts="skullbase" encoding="R">
 		0 1 0 0 1
		1 0 1 0 1
    </SkeletonInterpolator>
    <KeyPosition id="start" weight="1" time="0.0"/>
    <KeyPosition id="ready" weight="1" time="0.0"/>
    <KeyPosition id="strokeStart" weight="1" time="0.08829594"/>
    <KeyPosition id="stroke" weight="1" time="0.5"/>
    <KeyPosition id="strokeEnd" weight="1" time="0.5"/>
    <KeyPosition id="relax" weight="1" time="1.0"/>
    <KeyPosition id="end" weight="1" time="1.0"/>
  </ProcAnimation>
</bmlt:procanimationgesture>

Or linked by fileName. The fileName can be an URL (prefix with file:/ for files on your local system) or a file on the classpath (e.g. in the resource dir AsapRealizerDemo). Example:

<bmlt:procanimationgesture id="ani" fileName="file:/c:/procanimations/myprocanimation.xml"/>

<bmlt:transition>

Creates a transition animation. The start and end pose of the transition are determined automatically from its surrounding motions.

AttributeTypeUseDescription
class OpenSetItemrequiredJava class name of the transition used

Example: create a simple slerp transition on the arm's rotations, moving from the neutral pose toward the start arm pose of the conducting gesture

<bmlt:transition id="trans1" class="SlerpTransition" start="1" end="2">
  <bmlt:parameter name="joints" value="r_shoulder r_elbow r_wrist l_shoulder l_elbow l_wrist"/>
</bmlt:transition> 
<bmlt:procanimation id="beat1" name="3-beat" start="trans1:end"/>

<bmlt:blinkemitter>

Sets automatic blinking behavior with specified parameters. Waiting time between two automatic eye blinks is in the range [avgwaitingtime-range..avgwaitingtime+range]. Note: cannot have an end time (bmlt:blinkemitter is always persistent behavior). If automatic blink behavior was not turned on for the Virtual Human, bmlt:blinkemitter behaviors will be dropped. See ElckerlycDocumentation? under "Develop / VirtualHumanSpecs" for details on how to turn on automatic blink behavior. To change the parameters of the blinking behavior, you can either specify a new bmlt:blinkemitter behavior, which will override the previous one, or you can use a bmlt:parametervaluechange behavior (see example).

AttributeTypeUseDescription
avgwaitingtimenonnegative floatrequiredaverage waiting time between blinks. '0' means to turn off automatic blinking behavior.
rangenonnegative floatrequiredmaximum variation in waiting time

Example:

  <bmlt:blinkemitter id="blinkemitter1" start="0" range="0.5" avgwaitingtime="10"/>
  <bmlt:parametervaluechange id="pvc1" target="bml1:blinkemitter1" paramId="avgwaitingtime" start="20"> 
    <bmlt:trajectory type="instant" targetValue="1"/>
  </bmlt:parametervaluechange>

<bmlt:breathingemitter>

Sets automatic breathing behavior with specified parameters, animated by moving the shoulders slightly. Waiting time between two breaths is in the range [avgwaitingtime-range..avgwaitingtime+range]. Note: cannot have an end time (bmlt:breathingemitter is always persistent behavior). If automatic breathing was not turned on for the Virtual Human, bmlt:breathingemitter behaviors will be dropped. See ElckerlycDocumentation? under "Develop / VirtualHumanSpecs" for details on how to turn on automatic breathing. To change the parameters of the breathing behavior, you can either specify a new bmlt:breathingemitter behavior, which will override the previous one, or you can use a bmlt:parametervaluechange behavior (see example).

AttributeTypeUseDescription
avgwaitingtimenonnegative floatrequiredaverage waiting time between breaths. '0' means to turn off automatic blinking behavior.
rangenonnegative floatrequiredmaximum variation in waiting time

Example:

  <bmlt:breathingemitter id="breathingemitter1" start="0" range="2" avgwaitingtime="10"/>
  <bmlt:parametervaluechange id="pvc1" target="bml1:breathingemitter1" paramId="avgwaitingtime" start="20"> 
    <bmlt:trajectory type="instant" targetValue="1"/>
  </bmlt:parametervaluechange>

<bmlt:noise>

Generates a noise (typically Perlin noise) animation on a joint, specified by the type of noise, joint id and custom parameters.

AttributeTypeUseDescription
jointstringrequiredjoint id
type OpenSetItemrequiredtype of the noise

Example:

<bmlt:noise id="noise1" type="perlin" joint="vl5" start="0" end="100">
  <bmlt:parameter name="basefreqx" value="0.5"/>
  <bmlt:parameter name="baseamplitudex" value="0.05"/>
</bmlt:noise>

Note that Perlin noise behaviors have the following possible bmlt parameters: basefreqx, basefreqy, basefreqz, baseamplitudex, baseamplitudey, baseamplitudez, persistencex, persistencey, persistencez

<bmlt:facemorph>

Directly controls a face morph target defined on the avatar mesh.

AttributeTypeUseDescription
targetnamestringrequiredname of morph target
intensityfloatoptionalamnount to which the morph target is activated

Example:

<facemorph id="face1" targetname="Body_NG-mesh-morpher-Smile01-9" start="0" end="5"/>

BMLT description extensions

Most BMLT behaviors may also be used as a description extension for a core BML behavior. For example, procanimation can be used as description extension for gesture and controller can be used as description extension for posture.

<gesture start="3" id="beat1" type="LEXICALIZED" lexeme="conduct-3-beat" hand="RIGHT">
  <description priority="1" type="procanimation">
    <bmlt:procanimation id="beat1" name="conduct-3-beat" start="3">
      <bmlt:parameter name="a" value="5"/>
      <bmlt:parameter name="hand" value="RIGHT"/>
    </bmlt:procanimation>	
  </description>
</gesture>

BMLT also supports several speech description extensions, including SSML, Microsoft SAPI and various MaryTTS specifications.

Other Description Extensions Implemented by AsapRealizer

<speech> using Microsoft Speech API

Example:

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="1" type="application/msapi+xml">
      <sapi>I'm speaking <spell>BML</spell>.</sapi>
    </description>  	
  </speech>   
</bml>

<speech> using SSML

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="1" type="application/ssml+xml">
      <speak xmlns="http://www.w3.org/2001/10/synthesis">
      I'm <prosody pitch="high">speaking</prosody> BML.
      </speak>
    </description>  	
  </speech>   
</bml>

<speech> using Mary TTS

Find more information: UsingMaryTTS

<bml id="bml1">
  <speech id="speech1" start="5">   	
    <text>I'm speaking BML.</text>    	
    <description priority="10" type="maryxml">
      <maryxml xmlns="http://mary.dfki.de/2002/MaryXML">
      I'm speaking BML.
      </maryxml>
    </description>  	
  </speech>   
</bml>