wiki:MURML

Version 32 (modified by welberge, 7 years ago) (diff)

--

MURML

AsapRealizer's MURML extension encompasses the detailed description of facial animation and gesture. It does not encompas MURML's synchronization mechanisms and speech description, since these can be specified in BML and its core description levels. The namespace for MURML is  http://www.techfak.uni-bielefeld.de/ags/soa/murml.

MURML hand gesture

A hand gesture can be specified using the <murmlgesture> behavior.

<bml xmlns="http://www.bml-initiative.org/bml/bml-1.0"  id="bml1" > 
<murmlgesture id="gesture1" start="2" xmlns="http://www.techfak.uni-bielefeld.de/ags/soa/murml"> 
  <murml-description>    
    <parallel>
      <dynamic slot="HandLocation" scope="left_arm">
	<dynamicElement type="linear" scope="left_arm">
          <value type="start" name="LocLowerChest LocCCenter LocNorm"/>
          <value type="end" name="LocStomach LocPeripheryLeft LocFFar"/>
	</dynamicElement>
      </dynamic>
      <dynamic slot="HandShape" scope="left_arm">
        <dynamicElement>
            <value type="start" name="BSneutral"/>
            <value type="end" name="BSflat"/>
	</dynamicElement>
      </dynamic>
      <dynamic slot="ExtFingerOrientation" scope="left_arm">
	<dynamicElement>
            <value type="start" name="DirRA"/>
            <value type="end" name="DirALAA"/>
	</dynamicElement>
      </dynamic>
      <dynamic slot="PalmOrientation" scope="left_arm">
	<dynamicElement>
            <value type="start" name="PalmRU"/>
            <value type="end" name="PalmU"/>
	</dynamicElement>
      </dynamic>
    </parallel> 
  </murml-description>
</murmlgesture> 
</bml> 

or, preferably, using the MURML description extension:

<gesture id="g1" mode="LEFT_HAND" lexeme="BEAT" start="0.5"> 
  <description priority="1" type="murmlgesture"> 
    <murml-description xmlns="http://www.techfak.uni-bielefeld.de/ags/soa/murml"> 
    ...
    </murml-description> 
  </description> 
</gesture> 

MURML face gesture

MURML face gestures can similarly be described by the <murmlface> element

<bml xmlns="http://www.bml-initiative.org/bml/bml-1.0"  id="bml1" > 
<murmlface xmlns="http://www.techfak.uni-bielefeld.de/ags/soa/murml" id="face1" start="2"> 
<murml-description>
<dynamic> 
	<keyframing easescale="5" easeturningpoint="0.7"> 
		<phase> 
		<frame ftime="0.5"> 
		<posture>Humanoid (Body_NG-mesh-morpher-yeux_NG01-1 1 100)</posture> 
		</frame> 
		<frame ftime="1"> 
		<posture>Humanoid (Body_NG-mesh-morpher-yeux_NG01-1 1 0)</posture> 
		</frame> 
		<frame ftime="1.5"> 
		<posture>Humanoid (Body_NG-mesh-morpher-yeux_NG01-1 1 100)</posture> 
		</frame> 
		<frame ftime="2"> 
		<posture>Humanoid (Body_NG-mesh-morpher-yeux_NG01-1 1 0)</posture> 
		</frame> 
		</phase> 
	</keyframing> 
</dynamic> 
</murmlface> 
</murml-description>
</bml> 

Or preferably as an extension of e.g. <faceLexeme>:

<faceLexeme id="face1" start="2" lexeme="smile"> 
	<description priority="1" type="murmlface"> 
	<murml-description xmlns="http://www.techfak.uni-bielefeld.de/ags/soa/murml">	 
	<constraints>	 
	<dynamic> 
		<keyframing>			 
			<phase>			 
			<frame ftime="0.5"> 
			<posture>Humanoid (Body_NG-mesh-morpher-yeux_NG01-1 1 100)</posture> 
			</frame> 
			<frame ftime="1"> 
			<posture>Humanoid (Body_NG-mesh-morpher-yeux_NG01-1 1 0)</posture> 
			</frame>			 
			</phase> 
		</keyframing> 
	</dynamic> 
	</murml-description> 
	</description> 
</faceLexeme> 

From MURML to BML, an example

The example below shows how a MURML description can be translated to BML: MURML

<definition>
  <utterance>
    <specification>
    And now take <time id=“t1”/> this <time id=”t2”/> bar <time id=“t3”' chunkborder=“true”/>
    and make it <time id=“t4”/> this big. <time id=“t5”/>
    </specification>
    <focus onset=”t1” end=”t2” accent=”H*”/>
    <behaviorspec id=”gesture_1”>
      <gesture id=”pointing_to”>
        <affiliate onset=”t1” end=”t3”/>
        <param name=”refloc” value=”$Loc-Bar_1”/>
      </gesture>
    </behaviorspec>
    <behaviorspec id=”gesture_2”>
      <gesture>
        <affiliate onset=”t4” end=“t5”/>
        <constraints>
          <symmetrical dominant=”right_arm” symmetry=”SymMS”>
          <parallel>
            <static slot="HandShape" value="BSflat (FBround all o) (ThCpart o)"/>
            <static slot="PalmOrientation" value="DirL"/>
            <static slot="ExtFingerOrientation" value="DirA"/>
            <dynamic slot="HandLocation">
              <dynamicElement type="linear">
                <value type="start" name="LocShoulder LocCenterRight LocNorm"/>
                <value type="direction" name="DirR''/>
                <value type="distance" name="125.0''/>
              </dynamicElement>
            </dynamic>
          </parallel>
          </symmetrical>
        </constraints>
      </gesture>
    </behaviorspec>
  </utterance>
</definition>

BML

<bml id="bml1" xmlns="http://www.bml-initiative.org/bml/bml-1.0">
  <speech id="speech1"/>
    <text>
	And now take <sync id="t1"/> this <sync id="t2"/> bar
    </text>
    <description priority="1" type="maryxml">
      <maryxml xmlns="http://mary.dfki.de/2002/MaryXML">
        And now take <mark name="t1"/> <t accent="H*">this</t> <mark name="t2"/> bar
      </maryxml>    
    </description>
  </speech>  
  <pointing id="pointing_to" target="Loc-Bar_1" strokeStart="t1" strokeEnd="t3"/>
</bml>
<bml id="bml2" xmlns="http://www.bml-initiative.org/bml/bml-1.0" compositions="CHUNK_AFTER(bml1)">
  <speech id="speech1"/>
    <text>
    	and make it <sync id="t4"/> this big. <sync id="t5"/>
    </text>
  </speech>  
  <gesture id="gesture_2" strokeStart="t4" strokeEnd="t5" lexeme="SIZE" mode="BOTH_HANDS">
    <description priority="1" type="murmlgesture">
       <murml-description xmlns="http://www.techfak.uni-bielefeld.de/ags/soa/murml"> 
       <symmetrical dominant="right_arm" symmetry="SymMS">
         <parallel>
          <static slot="HandShape" value="BSflat (FBround all o) (ThCpart o)"/>
          <static slot="PalmOrientation" value="DirL"/>
          <static slot="ExtFingerOrientation" value="DirA"/>
          <dynamic slot="HandLocation">
            <dynamicElement type="linear">
              <value type="start" name="LocShoulder LocCenterRight LocNorm"/>
              <value type="direction" name="DirR"/>
              <value type="distance" name="125.0"/>
            </dynamicElement>
          </dynamic>
         </parallel>
       </symmetrical>
       </murml-description>
    </description>
  </gesture>
</bml>

The MURML example contains two chunks, one with the intonation phrase "And now take this bar" + gesture gesture_1 and one with the intonation phrase "and make it this big." + gesture gesture_2. In BML, these chunks can be modeled in two BML blocks that are composed using the CHUNK_AFTER composition. Since core-bml does not contain any specification mechanisms for the MURML focus, we model the focus in speech using the  Mary XML core description extension. The MURML pointing gesture (with id gesture_1) can directly be translated into a BML pointing gesture. gesture_2 is modeled in core BML as a lexicalized gesture. Its finer grained MURML specification is provided within a description extension.

MURML descriptions in detail

<murml-description>

The toplevel element of a murml description is always <murml-description>. It acts on the same level as <constraints> in  pre-BML MURML.

AttributeTypeUseDescription
idStringoptionalmurml-description id (currently not used)
pointStrokeboolean (true, false)optional, defaults to falsetrue if the gesture has its stroke at a specific point (rather than a stroke phase)
scopeStringoptionalthe scope of the gesture

Child elements: (<keyframing> | <dynamic> | <static> | <symmetrical> | <parallel> | <sequence>)

TODO: implement keyframing and sequence. Keyframing is supported when wrapped in dynamic.

<keyframing>

TODO

<dynamic>

AttributeTypeUseDescription
scopeString: typically neck, right_arm, left_armoptionalthe scope of the dynamic block
slotSlot: HandShape | HandLocation | PalmOrientation | ExtFingerOrientation | GazeDirection | Neckoptional

Child elements: ((dynamicElement+)|<keyframing>)

<static>

Specifies a single feature of a gesture that is to be held for a certain period of time.

AttributeTypeUseDescription
valueStringrequired
slotSlot: HandShape | HandLocation | PalmOrientation | ExtFingerOrientation | GazeDirection | Neckoptional
scopeString: typically neck, right_arm, left_armoptional
mode | pointTo optional, defaults to empty

TODO implement mode(?)

<symmetrical>

Enables a convenient way to specify a symmetrical gesture, namely, to define only one (the 'dominant') hand along with the specific symmetry constraints.

AttributeTypeUseDescription
dominant right_arm | left_arm required
symmetry SymM + any combination of up to three characters out of F|T|S (e.g: SymMS, SymMTF, SymMSFT) required

Child elements: (<parallel>)

<parallel>

Can comprise any combination of constraints, among which it imposes simultaneity.

Child elements: ((<dynamic>|<static>)+)

<sequence>

TODO: implement this

<dynamicElement>

AttributeTypeUseDescription
scopeString optional
typelinear | curve | chop optional

Child elements: (<value>+)

<value>

AttributeTypeUseDescription
typeString required
nameString required