wiki:UsingAsapRealizerWithIpaaca

Version 33 (modified by yaghoubzadeh, 6 years ago) (diff)

--

Using AsapRealizer with Ipaaca

AsapRealizer can be connected to Billie's rendering environment in Ogre through Ipaaca, can execute Ipaaca BML messages and can provide BML feedback using Ipaaca messages.

Preliminaries

  1. Setup the soa build script and create a top-level project dir as described here:  https://hiro.techfak.uni-bielefeld.de/twiki/bin/view/SociableAgents/SoaScript
  2. Install AsapRealizer as described in the QuickStart

Connecting Ipaaca to the Ogre rendering environment

  1. Clone billie_asap and soaresource
    soa clone-all billie_asap soaresource
    
  2. Build billie_asap; in the billie_asap dir:
    soa build-all
    
  3. Start spread:
    spread -n localhost
    
  4. Run billie from the resource dir, it internally unpacks billie from dist (TODO: how to setup the build system so that this is not needed??)
    ./go.billie
    
  5. Run the AsapRealizerDemoExternalRenderer demo in AsapDemo/AsapRealizerDemo/java:
    ant main #select asap.realizerdemo.AsapRealizerDemoExternalRenderer
    ant run
    

The externalrenderdemo makes use of the asapbillieipaaca.xml VirtualHumanLoader xml. This hooks the AnimationEngine and the FaceEngine up to a renderbody provided by Ipaaca:

<AsapVirtualHuman>   
...
<Loader id="ipaacaembodiment" loader="asap.ipaacaembodiments.loader.IpaacaEmbodimentLoader"/>
<Loader id="bodyfaceembodiment" requiredloaders="ipaacaembodiment" loader="asap.ipaacaembodiments.loader.IpaacaFaceAndBodyEmbodimentLoader">
  <renaming skeletonRenamingFile="billieskeletonrenaming.xml" morphRenamingFile="billiemorphsrenaming.xml"/>
</Loader>  
<Loader id="mixedskeletonembodiment" loader="hmi.physicsenvironment.MixedSkeletonEmbodimentLoader" requiredloaders="bodyfaceembodiment"/>
<Loader id="animationengine"
          loader="asap.animationengine.loader.MixedAnimationEngineLoader"
          requiredloaders="mixedskeletonembodiment,physicalembodiment">
    <GestureBinding basedir="" resources="Humanoids/billie/gesturebinding/" filename="gesturebinding.xml"/>
    <StartPose resources="Humanoids/billie/restposes/" filename="looselyhangingarms.xml"/>
    <Hns resources="Humanoids/shared/hns" filename="hns.xml"/>
</Loader>

<Loader id="faceengine" 
          loader="asap.faceengine.loader.FaceEngineLoader"
          requiredloaders="bodyfaceembodiment">
    <FaceBinding basedir="" resources="Humanoids/billie/facebinding/" filename="facebinding.xml"/>
</Loader>   
...
</AsapVirtualHuman>   

Submitting and receiving BML through Ipaaca

  1. Setup a VirtualHumanLoader xml file that uses an IpaacaToBMLRealizerAdapterLoader. An example configuration is provided in AsapDemo/AsapRealizerDemo/java/resource/asaparmandiaipaacaport.xml. To set up/configure your own use:
    <AsapVirtualHuman>
      <Loader id="realizer" loader="asap.realizerembodiments.AsapRealizerEmbodiment">
        ...
        <PipeLoader id="pipe" loader="asap.ipaacaadapters.loader.IpaacaToBMLRealizerAdapterLoader"/>
      </Loader>
      ...
    
  2. Start spread:
    spread -n localhost
    
  3. Run the AsapRealizerDemo with the newly defined xml
    ant run -Drun.argline="asaparmandiaipaacaport.xml"
    
  4. The realizer is now ready to receive and submit Ipaaca BML messages. The demo programs in java/src/asap/realizerdemo/AsapRealizerPortToIpaacaDemo.java and python/src/asap_realizerdemo/RealizerPortToIpaacaDemo.py show how to submit BML and receive BML feedback in java and python respectively.
  5. To start the python demo; in the python subdir: [TODO: this probably doesn't work without the python spread library installed.]
    ant resolve
    ant main #select asap_realizerdemo/RealizerPortToIpaacaDemo.py
    ant run
    
  6. To start the java demo; in the java subdir:
    ant main #select asap.realizerdemo.AsapRealizerPortToIpaacaDemo
    ant run
    

Connecting Ipaaca to the Ogre rendering environment AND Submitting and receiving BML through Ipaaca

Setup as above and run the AsapRealizerDemoExternalRenderer with

ant run -Drun.argline="asapbillieipaaca_ipaacaport.xml"

Connecting to an Ipaaca TTSAgent

Use the following TTS setup in your VirtualHumanLoader:

<Loader id="ttsbinding" loader="asap.ipaacattsbinding.loader.IpaacaTTSBindingLoader">
    <PhonemeToVisemeMapping resources="Humanoids/shared/phoneme2viseme/" filename="sampade2ikp.xml"/>
</Loader>

<Loader id="speechengine" loader="asap.speechengine.loader.SpeechEngineLoader" requiredloaders="facelipsync,ttsbinding">
   <Voice factory="WAV_TTS"/>
</Loader>

Examples config files for this include:

armandia_ipaacatts (armandia character, ipaaca tts, integrated ttsport, hmigraphics renderer)
asapbillie_ipaacatts (billie character, ipaaca tts, ogre renderer)
asapbillie_ipaacatts_ipaacaport (billie character, ipaaca tts, ipaacaport, ogre renderer)