Changes between Initial Version and Version 1 of BMLT

07/12/11 12:01:00 (8 years ago)



  • BMLT

    v1 v1  
     2= BML Twente = 
     3The BML Twente extension adds the specification of physical controllers, procedural animation, transition animations and synchronization to anticipated events to BML. It is used in the [[|Elckerlyc]] realizer. Here we present a quick overview of the BMLT tags and their syntax. The namespace for BMLT is, we use bmlt as the prefix for that namespace throughout this document. 
     5== The BMLT behavior elements == 
     7=== <bmlt:parameter> === 
     8Used to pass parameter values to other BMLT elements, such as <bmlt:controller>, <bmlt:procanimation>, <bmlt:keyframe> or <bmlt:transition>. 
     10||name       ||[[|OpenSetItem]]||required||parameter name|| 
     11||value      ||string     ||required||parameter value|| 
     13=== <bmlt:keyframe> === 
     14Describes a keyframe (or mocap) animation. Parameters can be passed using <bmlt:parameter>. The joints parameter selects the joints the animation should act upon. This is useful to subtract motion on a subsection of the body from full body mocap. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Mirror is always applied before joint selection. 
     16||name||[[|OpenSetItem]]||required||name of the keyframe animation|| 
     19<bmlt:keyframe id="v1" name="plain_rhand"> 
     20  <bmlt:parameter name="joints" value="r_shoulder r_elbow r_wrist"/> 
     21  <bmlt:parameter name="mirror" value="false"/> 
     25=== <bmlt:audiofile> === 
     26Behavior that plays an audio file. Useful to, e.g., play a sound effect when the avatar claps his hands. 
     28||filename||file||required||name of the file containing the audio|| 
     31<bmlt:audiofile id="v1" filename="c:/audio/clap.wav" start="3"/> 
     34=== <bmlt:controller> === 
     35Describes a physical controller behavior. Parameters can be passed to the controller using <bmlt:parameter>. 
     37||class||[[|OpenSetItem]]||required||Java class name of the controller|| 
     38Example: enable the balance controller with pelvis height at 1.3m (resulting in a balance motion with unbend legs) and slightly increased stiffness. 
     40<bmlt:controller id="balance1" class="BalanceController" start="0" end="10"> 
     41  <bmlt:parameter name="pelvisheight" value="1.3"/> 
     42  <bmlt:parameter name="stiffnessmultiplier" value="1.1"/> 
     46=== <bmlt:procanimation> === 
     47Describes a procedural animations. Parameters can be passed using <bmlt:parameter>. Like in keyframe animation, the joints parameter selects the joints the animation should act upon. The mirror parameter (default false) mirrors the animation in the mid-sagittal plane. Again, mirror is always applied before joint selection. 
     49||name||[[|OpenSetItem]]||required||name of the procedural animation|| 
     50Example: conduct a procedural 3 beat gesture with amplitude 5. 
     52<bmlt:procanimation id="beat1" name="3-beat" start="3"> 
     53  <bmlt:parameter name="a" value="5"/> 
     57=== <bmlt:transition> === 
     58Creates a transition animation. The start and end pose of the transition are determined automatically from its surrounding motions. 
     60||class||[[|OpenSetItem]]||required||Java class name of the transition used|| 
     61Example: create a simple slerp transition on the arm's rotations, moving from the neutral pose toward the start arm pose of the conducting gesture 
     63<bmlt:transition id="trans1" class="SlerpTransition" start="1" end="2"> 
     64  <bmlt:parameter name="joints" value="r_shoulder,r_elbow,r_wrist,l_shoulder,l_elbow,l_wrist"/> 
     66<bmlt:procanimation id="beat1" name="3-beat" start="trans1:end"/> 
     68=== <bmlt:blinkmodelchange> === 
     70Dennis, is this still up-to-date? 
     72Changes the parameters of the automatic blinking behavior. Waiting time between two automatic eye blinks is in the range [averagewaitingtime-range..averagewaitingtime+range]. 
     73Note: cannot have an end time (bmlt:blinkmodelchange is always persistent behavior). If automatic blink behavior was not turned on for the Virtual Human, bmlt:blinkmodelchange behaviors will be dropped. 
     75||averagewaitingtime||nonnegative float||required||average waiting time between blinks. '0' means to turn off automatic blinking behavior. || 
     76||range||nonnegative float||required||maximum variation in waiting time|| 
     79<bmlt:blinkmodelchange id="bmc1" start="5" averagewaitingtime="0.5" range="0.2"/> 
     82=== <bmlt:noise> === 
     83Generates a noise (typically Perlin noise) animation on a joint, specified by the 
     84type of noise, joint id and custom parameters. 
     86||joint||string||required||joint id|| 
     87||type||[[|OpenSetItem]]||required||type of the noise|| 
     89<bmlt:noise id="noise1" type="perlin" joint="vl5" start="0" end="100"> 
     90  <bmlt:parameter name="basefreqx" value="0.5"/> 
     91  <bmlt:parameter name="baseamplitudex" value="0.05"/> 
     95=== <bmlt:facemorph> === 
     97Dennis, is this still in use? 
     99=== <bmlt:interrupt> === 
     100The BMLT interrupt behavior provides the capability of specifying precisely when 
     101specific running or scheduled behaviors should be interrupted. 
     103||target||string||required||id of the target BML block to interrupt|| 
     104||include||comma separated list of strings||optional||set of behaviors to interrupt in the target BML block (unspecified is all)|| 
     105||exclude||comma separated list of strings||optional||set of behaviors NOT to interrupt in the target BML block (unspecified is none)|| 
     106Interrupt all behaviors in BML, except bml1:speech1 and bml1:gesture1. 
     108<bmlt:interrupt id="i1" target="bml1" start="shake1:stroke" exclude="speech1,gesture1"/> 
     111=== <bmlt:parametervaluechange> === 
     112The parametervaluechange behavior allows the modification of certain parameter values of ongoing behavior. 
     114||target||string||required||id of the behavior the parametervaluechange targets|| 
     115||paramId||string||required||id of the targeted parameter|| 
     117Change the volume of bml1:speech1: 
     119<bmlt:parametervaluechange target="bml1:speech1" paramId="volume" 
     120start="bml1:speech1:sync1" stroke="bml1:speech1:sync1+1"> 
     121<bmlt:trajectory type="linear" targetValue="90"/> 
     124The bmlt:parametervaluechange behavior contains a trajectory element that specifies the trajectory and value of the parameter that is changed. 
     126||targetValue||float||required||target value of the parameter|| 
     127||type||string||required||type of the trajectory (e.g. linear, instant) || 
     128||initialValue||float||optional||start value of the parameter, if omitted the start value is obtained from the targeted behavior|| 
     129The parametervaluechange behavior sets the parameter value on the target behavior according 
     130to the trajectory, in such a way that the targetValue is reached at the stroke 
     131of the parametervaluechange behavior. It is not allowed to constrain 
     132the end sync of the parametervaluechange behavior; it ends automatically when the target value is achieved. 
     134== Preplanning and activation == 
     135Scheduling a BML block typically takes a non-negligible amount of time, especially 
     136if the timing of speech is to be obtained through speech synthesis software. This 
     137is problematic for developing highly responsive virtual humans. BMLT provides 
     138pre planning as a mechanism to construct a behavior plan that can be activated later 
     139on. In a typical usage scenario of pre-planning, the SAIBA Behavior Planner already 
     140knows what behavior to execute, and wants to execute it (near) instantly later on, 
     141for example in reaction to some event such as an incoming response from the user.  
     142Preplanning is set up for a BML block, using the BMLT preplan attribute in that 
     143block. Preplanned BML blocks can be activated using an BMLT activate behavior. 
     144The preplanned BML block is activated as soon the activate behavior starts. 
     145Preplan bml1: 
     147<bml id="bml1" bmlt:preplan="true"> 
     151=== <bmlt:activate> === 
     152The bmlt:activate behavior activates a preplanned block. 
     154||target||string|required||id of the target block to activate|| 
     157<bmlt:activate id="a1" target="bml1"/> 
     162== Synchronization to Predicted Events ==   
     163We allow synchronization to external, possibly predicted events. This does not require an extension of BML, other than allowing a synchronization to reference ids that are not in the BML script. 
     164Example: link conducting beat 1, 2 and 3 of a procedural conducting to tick 1, 2 and 3 of a metronome. 
     166<bmlt:procanimation id="conduct1" name="3-beat"/> 
     167<constraint id="c1"> 
     168  <synchronize ref="conduct1:start"> 
     169    <sync ref="anticipators:metronome1:tick1"/>          
     170  </synchronize> 
     171  <synchronize ref="conduct1:beat2"> 
     172    <sync ref="anticipators:metronome1:tick2"/>          
     173  </synchronize> 
     174  <synchronize ref="conduct1:beat3"> 
     175    <sync ref="anticipators:metronome1:tick3"/>          
     176  </synchronize>   
     180== Persistent bmlt behaviors == 
     181Some BMLT behaviors are persistent (currently just bmlt:controller). They adhere to the same persistent patterns as gaze and posture in core BML. 
     183Specification of a persistent balance controller: 
     185<bml id="bml1" xmlns:bmlt=""> 
     186   <bmlt:controller id="balance1" class="BalanceController" start="1"/> 
     190== Mutually exclusive behavior using replacement groups == 
     192Each BML behavior can be assigned a replacement group using a bmlt:parameter. Elckerlyc ensures that at most one behavior of each replacement group is played at the same time. The played behavior is the behavior with the most recent start time.  
     196<bml id="bml1" xmlns:bmlt=""> 
     197  <bmlt:controller id="balance1" class="BalanceController"> 
     198    <bmlt:parameter name="pelvisheight" value="1.2"/> 
     199    <bmlt:parameter name="replacementgroup" value="balance"/> 
     200  </bmlt:controller> 
     201  <bmlt:controller id="balance1" class="BalanceController" start="4" end="8"> 
     202    <bmlt:parameter name="pelvisheight" value="0.8"/> 
     203    <bmlt:parameter name="replacementgroup" value="balance"/> 
     204  </bmlt:controller> 
     207Balances with stretched knees (pelvisheight = 1.2m) from time = 0 till time = 4, balances with bend knees (pelvisheight = 0.8m) from time = 4 till time = 8, then balances with stretched knees again. 
     209BMLT behaviors can overwrite persistent core bml behaviors by using their name as replacement group: 
     211<bml id="bml1" xmlns:bmlt=""> 
     212  <posture id="pose1" stance="standing" shape="open" part="lower"/>      
     213  <bmlt:controller id="balance1" class="BalanceController" start="4"> 
     214    <bmlt:parameter name="pelvisheight" value="0.6"/> 
     215    <bmlt:parameter name="replacementgroup" value="posture"/> 
     216  </bmlt:controller> 
     220== BMLT description extensions == 
     221Most BMLT behaviors may also be used as a description extension for a core 
     222BML behavior. For example, procanimation can be used as description extension 
     223for gesture and controller can be used as description extension for posture. 
     225<gesture start="3" id="beat1" type="LEXICALIZED" lexeme="conduct-3-beat" hand="RIGHT"> 
     226  <description priority="1" type="bmlt:procanimation"> 
     227    <bmlt:procanimation id="beat1" name="conduct-3-beat" start="3"> 
     228      <bmlt:parameter name="a" value="5"/> 
     229      <bmlt:parameter name="hand" value="RIGHT"/> 
     230    </bmlt:procanimation>        
     231  </description> 
     234BMLT also supports several speech description extensions, 
     235including SSML, Microsoft SAPI and various MaryTTS specifications. 
     238== Other Description Extensions Implemented by Elckerlyc == 
     239=== <speech> using Microsoft Speech API === 
     242<bml id="bml1"> 
     243  <speech id="speech1" start="5">        
     244    <text>I'm speaking BML.</text>       
     245    <description priority="1" type="application/msapi+xml"> 
     246      <sapi>I'm speaking <spell>BML</spell>.</sapi> 
     247    </description>       
     248  </speech>    
     252=== <speech> using SSML === 
     254<bml id="bml1"> 
     255  <speech id="speech1" start="5">        
     256    <text>I'm speaking BML.</text>       
     257    <description priority="1" type="application/ssml+xml"> 
     258      <speak xmlns=""> 
     259      I'm <prosody pitch="high">speaking</prosody> BML. 
     260      </speak> 
     261    </description>       
     262  </speech>    
     266=== <speech> using Mary TTS === 
     268<bml id="bml1"> 
     269  <speech id="speech1" start="5">        
     270    <text>I'm speaking BML.</text>       
     271    <description priority="10" type="maryxml"> 
     272      <maryxml xmlns=""> 
     273      I'm speaking BML. 
     274      </maryxml> 
     275    </description>       
     276  </speech>    
     281== BMLT Feedback == 
     282In addition to the feedback specified in core BML, Elckerlyc provides global timestamps in all its feedback messages and additionally a local time stamp (time in seconds since start of the BML performance) for Sync-Point Progress Feedback. Elckerlyc also provides information on the planning status of a BML request, using Planning Start and Planning Finished feedback. 
     285=== Scheduling Start Feedback === 
     286Notifies the behavior planner that scheduling of a requested BML performance has begun. 
     288The message includes: 
     289  * A character identifier 
     290  * A BML block identifier 
     291  * A global timestamp 
     292  * The predicted start time of the BML block 
     294=== Scheduling Finished Feedback === 
     295Notifies the behavior planner that scheduling of a requested BML block is finished. 
     297The message includes: 
     298  * A character identifier 
     299  * A BML request identifier 
     300  * A global timestamp 
     301  * The predicted start time of the BML block 
     303== The BMLT BML attributes == 
     304=== append-after === 
     305append-after(X) scheduling attribute instructs the Realizer to execute the new BML 
     306block immediately after all behaviors from the prior BML blocks on list X have finished 
     309<bml id="bml4" scheduling="append-after(bml2,bml3)"> 
     313=== allowexternalrefs === 
     314The allowexternalrefs attribute is used to indicate that a BML block may contain 
     315time constraints that refer to behaviors in other (external) BML blocks. Such 
     316references are of the form bmlid:behaviorid:syncid. 
     319<bml id="bml2" bmlt:allowexternalrefs="true"> 
     320<bmlt:parametervaluechange id="pvc1" target="bml1:speech1" 
     321paramId="volume" start="bml1:speech1:sync1" stroke="bml1:speech1:end"> 
     322<bmlt:trajectory type="linear" initialValue="0" targetValue="100"/> 
     326=== The interrupt shorthand === 
     327The interrupt attribute is a shorthand for the SAIBA Behavior Planner to remove a 
     328selected set of BML blocks from the multimodal behavior plan before scheduling the 
     329content of the BML block it is in. 
     332<bml id="bmlNew" bmlt:interrupt="bml1,bml2,..,bmln"> 
     333bmlNew content 
     336This example is a shorthand for:\\ 
     3371. Send a BML block to the Realizer that interrupts bml1..bmln: 
     339<bml id="bmlInterrupt"> 
     340  <bmlt:interrupt id="interrupt1" target="bml1"/> 
     341  <bmlt:interrupt id="interrupt2" target="bml2"/> 
     342  .. 
     343  <bmlt:interrupt id="interruptn" target="bmln"/> 
     3462. Wait for block end feedback of bmlInterrupt (to make sure bml1..bmln are properly removed from 
     347the multimodal behavior plan). 
     3483. Send the new BML block bmlNew: 
     350<bml id="bmlNew"> 
     351bmlNew content 
     354=== The onStart shorthand === 
     355The onStart attribute is a shorthand for the SAIBA Behavior Planner to activate a 
     356selected set of BML blocks in the multimodal behavior plan whenever a certain BML block starts. 
     359<bml id="bmlNew" bmlt:onStart="bml1,bml2,..,bmln"> 
     360  bmlNew content 
     363This is a shorthand for: 
     365<bml id="bmlActivate"> 
     366  <bmlt:activate id="activate1" target="bml1"/> 
     367  <bmlt:activate id="activate2" target="bml2"/> 
     368  .. 
     369  <bmlt:activate id="activaten" target="bmln"/> 
     370  bmlNew content