Version 47 (modified by welberge, 6 years ago) (diff)


Using AsapRealizer with Ipaaca

AsapRealizer can be connected to Billie's rendering environment in Ogre through Ipaaca, can execute Ipaaca BML messages and can provide BML feedback using Ipaaca messages.


  1. Prepare your system as described in RecommendedSystemSetup
  2. Setup SoaScript
  3. Install AsapRealizer as described in the QuickStart

Connecting Ipaaca to the Ogre rendering environment

  1. Clone billie_asap and soaresource
    soa clone-all billie_asap soaresource
  2. Build billie_asap; in the billie_asap dir:
    soa build-all billie_asap
  3. Start spread:
    spread -n localhost
  4. Run billie from the billie_asap/resource dir, it internally unpacks billie from dist
  5. Run the AsapRealizerDemoExternalRenderer demo in AsapDemo/AsapRealizerDemo/java:
    ant main #select asap.realizerdemo.AsapRealizerDemoExternalRenderer
    ant run

The externalrenderdemo makes use of the asapbillieipaaca.xml VirtualHumanLoader xml. This hooks the AnimationEngine and the FaceEngine up to a renderbody provided by Ipaaca:

<Loader id="ipaacaembodiment" loader="asap.ipaacaembodiments.loader.IpaacaEmbodimentLoader"/>
<Loader id="bodyfaceembodiment" requiredloaders="ipaacaembodiment" loader="asap.ipaacaembodiments.loader.IpaacaFaceAndBodyEmbodimentLoader">
  <renaming skeletonRenamingFile="billieskeletonrenaming.xml" morphRenamingFile="billiemorphsrenaming.xml"/>
<Loader id="mixedskeletonembodiment" loader="hmi.physicsenvironment.MixedSkeletonEmbodimentLoader" requiredloaders="bodyfaceembodiment"/>
<Loader id="animationengine"
    <GestureBinding basedir="" resources="Humanoids/billie/gesturebinding/" filename="gesturebinding.xml"/>
    <StartPose resources="Humanoids/billie/restposes/" filename="looselyhangingarms.xml"/>
    <Hns resources="Humanoids/shared/hns" filename="hns.xml"/>

<Loader id="faceengine" 
    <FaceBinding basedir="" resources="Humanoids/billie/facebinding/" filename="facebinding.xml"/>

Using IpaacaWorldEnvironment (e.g. to point to/look at objects)

On the AsapRealizer side:

WorldObjectEnvironment we = new WorldObjectEnvironment();
IpaacaWorldEnvironment ipWEnv = new IpaacaWorldEnvironment(we.getWorldObjectManager());
List<Environment> environments = new ArrayList<Environment>();
AsapEnvironment ee = new AsapEnvironment();
ee.init(environments, ..

On the OgreRender side: Add

Billie	SceneInformation	soaOgreSceneInformationComponent

to resource/plugins.conf

Submitting and receiving BML through Ipaaca

With Billie

Start AsapRealizer with

ant run -Drun.main.class="asap.realizerdemo.AsapRealizerDemoExternalRenderer" -Drun.argline="asapbillieipaaca_ipaacaport.xml"

This moves Billie in the restpose when succesfully started.

In a new console window: send a BML file, wait for it to finish:

ant run -Drun.main.class="asap.realizerdemo.IpaacaBMLSender" -Drun.argline="simpledemo/helloworld2.xml"

In general

  1. Setup a VirtualHumanLoader xml file that uses an IpaacaToBMLRealizerAdapterLoader. An example configuration is provided in AsapDemo/AsapRealizerDemo/java/resource/asaparmandiaipaacaport.xml. To set up/configure your own use:
      <Loader id="realizer" loader="asap.realizerembodiments.AsapRealizerEmbodiment">
        <PipeLoader id="pipe" loader="asap.ipaacaadapters.loader.IpaacaToBMLRealizerAdapterLoader"/>
  2. Start spread:
    spread -n localhost
  3. Run the AsapRealizerDemo with the newly defined xml
    ant run -Drun.argline="asaparmandiaipaacaport.xml"
  4. The realizer is now ready to receive and submit Ipaaca BML messages. The demo programs in java/src/asap/realizerdemo/ and python/src/asap_realizerdemo/ show how to submit BML and receive BML feedback in java and python respectively.
  5. To start the python demo; in the python subdir: [TODO: this probably doesn't work without the python spread library installed.]
    ant resolve
    ant main #select asap_realizerdemo/
    ant run
  6. To start the java demo; in the java subdir:
    ant main #select asap.realizerdemo.AsapRealizerPortToIpaacaDemo
    ant run

Connecting Ipaaca to the Ogre rendering environment AND Submitting and receiving BML through Ipaaca

Setup as above and run the AsapRealizerDemoExternalRenderer with

ant run -Drun.argline="asapbillieipaaca_ipaacaport.xml"

Connecting to an Ipaaca TTSAgent

Use the following TTS setup in your VirtualHumanLoader:

<Loader id="ttsbinding" loader="asap.ipaacattsbinding.loader.IpaacaTTSBindingLoader">
    <PhonemeToVisemeMapping resources="Humanoids/shared/phoneme2viseme/" filename="sampade2ikp.xml"/>

<Loader id="speechengine" loader="asap.speechengine.loader.SpeechEngineLoader" requiredloaders="facelipsync,ttsbinding">
   <Voice factory="WAV_TTS"/>

Examples config files for this include:

armandia_ipaacatts (armandia character, ipaaca tts, integrated ttsport, hmigraphics renderer)
asapbillieipaaca_ipaacatts (billie character, ipaaca tts, ogre renderer)
asapbillieipaaca_ipaacatts_ipaacaport (billie character, ipaaca tts, ipaacaport, ogre renderer)