Songs in 3D: Making a Simple X3D Sequencer

All musicians who work with modern instruments and recording gear are familiar with sequencing. We use devices called sequencers to coordinate different instruments or sound modules, lights and other effects during a performance. Where once these were standalone and pricey hardware devices, most sequencers today are software on commercial grade home computers. The midi player that you have on your home computer is giving instructions to one or more software or hardware sound synthesizers to play a music and can even be used in some systems to control lights. The act of sequencing events by automatic control is carried out by a sequencer. This tutorial presents simple means for creating a software sequencer in your VRML or X3D worlds.

To see the model described in this tutorial, you will need Flux Viewer installed. To view the model, go to this demo page. Wait for the download to complete. You will receive the usual warning from Internet Explorer if you use the IE browser about running Active-X objects. Have no fear! Hit the yellow box and keep going.

Sequencing Basics

Sequencing is a common concept in media presentation. The basic notion is that a single time clock controls the order of multiple events. Contrast this to real-time sequencing in which event values determine when other events occur. In real-time, for example, you can use the value of a sound beginning (startTime) or its active or inactive status (isActive = TRUE | FALSE) to control other events. In VRML/X3D, we do this by routing event values through eventIn/Out interfaces (in X3D, accessTypes) directly or through scripts using the directOut capability.

A real-time sequence can be very complex and almost non-deterministic, that is, varies by multiple conditions as to when an event occurs or if it occurs at all. The event cascades of VRML can simulate conditions complex enough that the resulting behavior can be considered emergent for you chaos theorists. This isn't bad. Unexpected behavior in content is delightful. Unexpected surprises in program code are not. That is the difference between serendipity and bugs.

Simple sequencing is not like that. In a simple sequencer, a script and a single clock are combined to control precisely when events occur. A simple sequencer may actually be doing two tasks:

1. Creating a linear sequence of events, that is, a list of events occur in list order


2. Ordering some events to occur at the same time

It can be convenient to think of this as music time where all instruments are playing together and get to the beginning or end of a phrase at the same time. In short, in a simple sequence, the idea is to get rid of any unpredictability. This is Multimedia 101.

As in music, a single medium, multiple media or multimedia often uses simple sequencing. You see it in combinations of music and dance where certain dance steps must occur on certain beats of a musical phrase, in theatre when certain lights are turned on or off as an actor speaks a certain line or sound effects are played, and even in office presentations in PowerPoint where animated text effects are added to emphasis bullet points (an overused and often distracting abuse of PowerPoint, but I digress).

In VRML/X3D, simple sequencing is well… simple. This tutorial provides an example.

Sequencer Basics

A simple VRML/X3D sequencer consists of three main components:

1. A TimeSensor where the cycleInterval sets the basic sequencing time unit, say a second and is looping to create a continuous stream of timestamps, each one second in duration


2. A script that activates the object behaviors that are to be sequenced and synchronized by receiving clock timestamps and activating behaviors at specific increments of the clock


3. The objects and behaviors that are sequenced and synchronized

A VRML scene may have multiple simple sequences overlapping or occurring independently. The browser takes cares of ensuring all of these are coordinated from frame to frame. Multiple sequencers might be using the same objects. However, this is very tricky. It is difficult to ensure multiple sequencers aren’t sending timing events to the same objects at exactly the same timestamp, thus resulting in a condition called multiple fan-in.

When multiple events are sent to the same eventIn/accessType at the same timestamp, browser behavior becomes unpredictable. If the objects are shared by multiple sequencers, make certain two sequencers are not activated at the same or in overlapping times. If you build combos where real-time events activate sequencers, it is easy to fall into that trap. Remember the prime pragma of the Goddess XimPlicity: KISS. KISS.

The example provided in this tutorial is an animated band playing a song I wrote and recorded with my band, Ground Level Sound.

The opening scene looks like this when it loads. The user clicks on the rotating rose to start the song and the sequence.



Like karaoke displays, one wants to hear the music and see the lyrics displayed as the song plays, and for fun, present animations that illustrate the lyrics, show the album cover, give credits, etc. In good MTV fashion, we also want the cameras to move to different positions. For this we need one sequencer coordinating:

1. Text displays, song lyrics in this example


2. Image displays, images that illustrate the song or are animation for the presentation, analogous to theatre backdrops


3. Cameras selected

The behavior of the animated band is controlled by the Media Machines Flux VRML/X3D extension, the AudioSensor routed to orientation interpolators for the H-anim avatar musicians. This model was created by Keith Victor and other than reskinning the objects to give a different fanciful look, these models are left as Keith made them. To be truer to the song, I need to add more musicians, but for this example, three are sufficient. This is an example of a sequencer being one part of a more complex scene.

NOTE: Because this example uses the MM AudioSensor and Keith’s model, this example only works in MediaMachines Flux Viewer. However, the sequencer will work in any VRML/X3D browser.

NavigationInfo Element

In contrast to River of Life (ROL), this scene is explicitly not a free roamer. The user is expected to load it, start it, and watch it. Bang! Done! This means the NavigationInfo element type attribute is set to None to disable movement with the cursor.


<NavigationInfo DEF='DefaultNav' type='NONE' />


Since there is no movement, other attributes of this element are left unspecified.

Background Elements

To begin putting in the resources we want to sequence, we specify three background nodes. The first is automatically bound by the browser on loading the world. Backgrounds cannot be animated except to bind and unbind them in scripts or using event routing. The ability to change them in scripts can be useful when changing the mood or setting of a scene.


<Background DEF='Background1'
backUrl='photos/the-third-planet.jpg'
frontUrl='photos/the-third-planet.jpg'
leftUrl='photos/the-third-planet.jpg'
rightUrl='photos/the-third-planet.jpg'
bottomUrl=''
/>

<Background DEF='Background2'
backUrl='texture/stars.jpg'
frontUrl='texture/stars.jpg'
leftUrl='texture/stars.jpg'
rightUrl='texture/stars.jpg'
topUrl='texture/stars.jpg'
bottomUrl='texture/stars.jpg'

/>

<Background DEF='Background3'
backUrl='texture/skull.jpg'
frontUrl='texture/skull.jpg'
leftUrl='texture/skull.jpg'
rightUrl='texture/skull.jpg'
topUrl='texture/skull.jpg'
bottomUrl='texture/skull.jpg'
/>


The first background (a sci-fi planet) is automatically loaded with the scene. The second (among the stars) is bound by the sequencing script during the solo guitar sequence. The third (old Harry) is bound at the closing shot while the record label logo is displayed.

Animated Backdrops

Animated backdrops are simple one sided index face set rectangles with Movie textures (gif images in this case) or image textures, typically jpegs and pngs for images with pixel transparency. Flux Viewer supports gif animation in the Movie Texture elements but does not support gif images in the Image Texture elements. It does support PNG images and PNGs do support transparency.

Side note: An easy way to make transparencies is the use Microsoft Photo editor in combination with PaintBrush. Both save PNG formats and are cheap!

In this example, there are two backdrops: a foreground and a background. The foreground is used in combination with the lyric text to animate the song lyrics or for effects during guitar solos. It is smaller and just behind the band. The background is a largish animated image that is further back and runs more or less constantly with a flame animation until turned off for the guitar solo Here is the look with the album cover and song title.



As the song plays, the images and the lyrics are switched in synchronization with the song. Here is the scene after the guitar solo.



Yes, I am a Drew Barrymore fan. Who isn’t?

The main animated backdrop is the flame effect seen behind Drew. Here is the code:

<Transform DEF='FlameBackground' translation='0 0 1' scale='20 20 1'>
<Shape containerField='children'>
<Appearance containerField='appearance'>
<MovieTexture DEF='MovieTexture_Flame' containerField='texture'
loop='true' url='"movies/flame.gif"'/>
<Material DEF='FlameMat' containerField='material' ambientIntensity='0.200'
shininess='0.200' transparency='1.000' diffuseColor='.8 .8 .8'/>
</Appearance>
<IndexedFaceSet containerField='geometry' solid='false' creaseAngle='0.524'
coordIndex='0 1 2 -1 0 2 3 -1'
colorIndex='0 0 0 -1 0 0 0 -1'>
<Coordinate containerField='coord'
point='-1 1 0 -1 -1 0 1 -1 0 1 1 0'/>
<TextureCoordinate containerField='texCoord' point='' />
<Color containerField='color' color=' .801 .801 .801'/>
</IndexedFaceSet>
</Shape>
</Transform>


The size of the backdrop can vary and of course, this is just a textured face so it can be combined with other faces in various positions, sizes, angles and can be in a Switch. In this example, I alternate the background with the backdrop using the material element transparency attribute in combination with binding the background. You can see this effect in the sweep camera sequence.

The advantage of this sequence is while the background is a box textured on the inside and all six faces, the backdrop is a 2 dimensional one sided rectangle. When the camera moves in 3D space, the edges of the backdrop become obvious and the illusion is broken. By making the backdrop invisible and changing the backdrop to a semi-contiguous image (the stars), the effect is both dramatic, hides the edges of the backdrop and blends better at the edges of the background. The original sci-fi background doesn’t blend so that distortion is also disguised. The image in the foreground is switched to an animated lightning adding a little more dramatic distraction to the solo camera sweep sequence.

All stupid pet tricks but as Mick Jagger says, “It’s all for the show, ya know.”

Viewpoints

The Viewpoints to be animated are three plus there is an opening viewpoint. There is a camera on the left side of the scene, a camera on the right, and an animated camera that moves in a circle around the scene called a ‘sweep camera’. Viewpoints in X3D and VRML97 are almost identical with the exception of the containerField attribute. This is the opening viewpoint.


<Viewpoint DEF='Viewpoint1'
containerField='children'
description='Viewpoint1'
jump='true'
fieldOfView='0.785'
position='0 1.03 2.8'
orientation='0 0 1 0'/>


Cameras are animated using the same techniques for animation of all objects, that is combinations of timers and interpolators such as Position Interpolators and Orientation Interpolators.


<PositionInterpolator DEF='AnimCamera_pos0'
key='
0 .3401 .69799 1'
keyValue='
0 1.03 2.8
.74343 2.17894 1.89748
-2.11302 2.48144 1.9862
0 1.03 2.8'/>


The values of the interpolators can be routed to the position attributes of the Viewpoint elements


<ROUTE fromNode='AnimCamera' fromField='fraction_changed' toNode='AnimCamera_EIEO' toField='set_fraction'/<
<ROUTE fromNode='AnimCamera_EIEO' fromField='modifiedFraction_changed' toNode='AnimCamera_pos0' toField='set_fraction'/>


or to the translation attributes of Transform elements that contain the Viewpoints. The difference is one of how the interpolator treats spatial change. For example, sending the output of an Orientation Interpolator directly to the Viewpoint position attribute causes the camera to turn in place. Sending it to a parent transform causes it to move in a circular motion around the center of the space defined by the transform, thus the term, a sweep camera.

NOTE: Look carefully at the attribute names of the ROUTEs above and notice that they changed from their names in VRML97. This is one of the changes made in the XMLization of VRML that are an annoyance when working in both versions of the language.

This is the look of the sweep camera with the background rebound to a star field to achieve the effects described above. You can see a little of the lightning effect from the foreground backdrop in the upper left corner.

Time Sensor

The time sensor used is quite basic. It provides a one second cycle that is fed continuously to the sequencing script. It is the script that does all of the work of sequencing.


<TimeSensor DEF='Sequence_Timer'
containerField='children'
cycleInterval='1'
loop='TRUE'
startTime='-1'/>


You might think as I did that you will need a sophisticated set of timers to coordinate these events given that timers issues SFTime events (timestamps) and the means to change text nodes and images is an X3D Switch element which takes SFInt32 (integers) in the whichChoice field. However as we shall see next, this is really as simple as having a list of events and some basic fields in a Script. This design relies on the fact that the cycleInterval value is a second and a second is a small enough granularity for our purpose here.

As long as you don’t need sub-second timing, this technique works. If you need sub-second timing, you will want to look at a more sophisticated technique such as combining a timer and a scalar interpolator. If you feed the results directly back to the Switch elements, this technique requires you to take the output of the scalar interpolator and convert it back to integers that correspond to the division of the second into the fractional values required (eg, tenths of a second). There is a PROTO in the ROL tutorial that does something similar to this. Fortunately, using the script technique, as we shall see, even that isn’t necessary.

The Switches

The easiest way to coordinate a sequenced series is to put the elements/nodes inside a switch node. A switch is a simple element. It has a whichChoice field with integer values (SFInt32) starting at zero, and contains VRML/X3D elements that are the contents of the switch. These are automatically numbered in their order of occurrence. The first element is element 0, the next element is the element 1, the element after that is element 2 and so on. The browser displays the element corresponding to the integer value in the whichChoice field. If whichChoice is zero, the first element is displayed, if it is 3, the third element is displayed and so on. I leave the 0 element string field blank because a switch displays the choice specified when the scene loads, so if you want that to be blank, you need a node with an empty string.

NOTE: Remember that in XML, if you use apostrophes or other reserved characters in a text value, you need to use the XML character entity values for these. For example, the line “So don’t expect me to come” has to be entered as follows:


string=’”So don't expect me to come”’


So, a switch is a list of nodes and the whichChoice value is the index into that list.


<Switch DEF='TakingMyTimeLyric' containerField='children' whichChoice='0'>

<!-- Line 0: Blank -->

<Group containerField='children'>
<Transform containerField='children'>
<Shape containerField='children'>
<Appearance containerField='appearance'>
<Material DEF='WhiteFont' containerField='material' ambientIntensity='0.200'
shininess='1.000' diffuseColor='1 0 0'/>
</Appearance>
<Text containerField='geometry' string='""' maxExtent='0.000'>
<FontStyle DEF='Poem' containerField='fontStyle' family='SERIF'
style='ITALIC' justify='"MIDDLE" "BEGIN"' size='0.2400'
spacing='1.000'/>
</Text>
</Shape>
</Transform>
</Group>

<!-- Line 1: Title: Taking My Time -->

<Group containerField='children'>…


When another element such as a script sets the switch by sending an integer event, the switch element automatically displays the corresponding element. This can be used for different purposes such as flipbook style animation. In this example, it is used to display the images and lyrics of the song. The sequencing script receives the timing events and based on the value received, sets the switch value. The switch for the foreground backdrop images is the same structure with the exception that the elements are groups containing Image and Movie textures instead of text nodes. Otherwise, a switch is a switch.


<!-- Image 15 -->

<Group containerField='children'>
<Transform scale='2 2 1' containerField='children'>
<Shape containerField='children'>
<Appearance containerField='appearance'>
<ImageTexture containerField='texture' url='"photos/argument2.jpg"'/>
<Material DEF='material4' containerField='material' ambientIntensity='0.200'
shininess='0.200' diffuseColor='.801 .801 .801'/>
</Appearance>
<IndexedFaceSet containerField='geometry' solid='false' creaseAngle='0.524'
coordIndex=' 0 1 2 -1 0 2 3 -1'
texCoordIndex='0 1 2 -1 0 2 3 -1'>
<Coordinate containerField='coord' point='-1 1 0 -1 -1 0 1 -1 0 1 1 0'/>
<TextureCoordinate containerField='texCoord' point='0 1 0 0 1 0 1 1'/>
</IndexedFaceSet>
</Shape>
</Transform>
</Group>


Note that the display order of elements correspond only to the integer index received in the whichChoice field of the switch meaning they do not have to displayed in the order of the list. The sequencing script controls the display order. The order in the switch element sets their whichChoice integer value only. You can think of this as a numbered list or as an index into an array.

This is very handy because one may mix the switch elements in different orders and display some more than once as I do. For that reason, it is a good idea to name them with comments plus their integer value in the switch stack. This helps enormously when you are scripting the sequence because you are like to change the display order several times and you only want to change the choice sent to the switch, not reorder the script or the switch.

In the Switch for the lyrics, I also add the lyric line corresponding to the switch value in the comment. This is incredibly useful because while you will not be changing this order necessarily, you need to get the timing values for the script and you need to put those where you can find them. We’ll see next in the script how that works.

X3D Script Headers

An X3D script like a VRML script has a header of fields followed by a JavaScript with functions. There are some subtle syntax differences so pay attention to this example. The X3D XML script element uses an XML CDATA markup container and this removes the need for the quotation marks and the url flag used in VRML. Where VRML has fields and events flags, the X3D header just has fields but they have accessTypes and subelements. Where a VRML script header looks like this


DEF Flamescript Script {
eventIn SFBool vanish
field SFNode mat USE FlameMat
directOutput TRUE
mustEvaluate FALSE
url "javascript:


an X3D Script header looks like this:


<Script DEF='FLAMESCRIPT'
directOutput='TRUE'
mustEvaluate='FALSE'>
<field name='vanish' accessType='inputOnly' type='SFBool'/>
<field name='mat' accessType='initializeOnly' type='SFNode'>
<Material
containerField='value'
USE='FlameMat'/>
</field>
<![CDATA[javascript: …


The difference in verbosity and complex look isn’t lost on the casual observer. That’s XML. Caveat vendor.

You will note that X3D Scripts use JavaScript just as VRML97 does. Mercifully, the X3D designers could not change JavaScript in function or syntax, so it works exactly as it did in VRML97. The Script node itself, changed in names and possibly functionality although I haven’t grokked it completely yet.

The Sequencing Script

Now we come to the nut of this tutorial: the sequencing script. This is really much simpler than I expected it to be when I started this prototype. This combination of Script, timers and switch could be combined into a proto (an exercise left to the reader ;-)).

The sequencing JavaScript is a single function. It is conceptually simple but takes some time to build because you need the timing values in seconds for each line of lyric and the effects (backdrop switching, camera animation) corresponding to the song. Unfortunately, this is where (Cheap = = Tedious). Welcome to Early Adopter Karma, aka, the consequences of doing things manually. To get the timing values of the song, that is, how many seconds into the song does the lyric happen, you can guess the values using your watch while the song plays.

Alternatively, if you have a recording program such as Adobe Audition or Cool Edit, you can play the sound file to the position where the lyric line is, pause the play, write down the seconds value in the timer readout of the program, and move to the next line. This is why it pays to comment the song lyrics into the switch elements before you write the sequencing script. You need those lyric whichChoice values for the script and you have to map those to the timing values. Then you can run the X3D animation and make adjustments.

Let’s look at the sequencer script and everything becomes clear (if not, take up knitting).

The script header contains the event interface fields for the timer values and switch whichChoice values followed by the reference elements for values in the scene set by the sequencer, that is, the viewpoints and backgrounds using their respective set_bind events.


<Script DEF='Kareoke' directOutput='FALSE' mustEvaluate='FALSE'>
<field name='cycleTime' accessType='inputOnly' type='SFTime'/>
<field name='whichChoiceLyric' accessType='outputOnly' type='SFInt32'/>
<field name='whichChoiceImage' accessType='outputOnly' type='SFInt32'/>
<field name='curCycle' accessType='inputOutput' type='SFInt32' value='0'/>

<field name='kareokeTS' accessType='initializeOnly' type='SFNode'>
<TimeSensor USE='Kareoke_Timer'/>
</field>
<field name='viewpointGen' accessType='initializeOnly' type='SFNode'>
<TimeSensor USE='Generated_Viewpoint'/>
</field>
<field name='viewpointTaking' accessType='initializeOnly' type='SFNode'>
<TimeSensor USE='TakingMyTimeView'/>
</field>
<field name='viewpointGen0' accessType='initializeOnly' type='SFNode'>
<TimeSensor USE='Generated_Viewpoint0'/>
</field>
<field name='sweepcam0' accessType='initializeOnly' type='SFNode'>
<TimeSensor USE='Moving_Viewpoint'/>
</field>
<field name='background1' accessType='initializeOnly' type='SFNode'>
<Background USE='Background1'/>
</field>
<field name='background2' accessType='initializeOnly' type='SFNode'>
<Background USE='Background2'/>
</field>
<field name='background3' accessType='initializeOnly' type='SFNode'>
<Background USE='Background3'/>
</field>
<field name='mat' accessType='initializeOnly' type='SFNode'>
<Material USE='FlameMat'/>
</field>

<![CDATA[javascript:


The function is based on receiving timestamps in intervals of a second. Since the received value is an SFTime event and the switch whichChoice is an SFInt32, the script has to provide the integer values to the scripts. Since we are using seconds in integers, the SFTime values are counted and a cycle is incremented as each SFTime event is received. The cycle value is then tested in the if/else statement. As the desired cycle value is created by the increment statement, the appropriate whichChoice value is sent to the appropriate switch. It is the combination of the timestamp, the incrementing of the cycle value, and the whichChoice integers that creates the synchronized sequence.

Here is a commented fragment of the function:


function cycleTime (value) {

The value is timestamp received from KareokeTS. As each value is received, the curCycle value is incremented by one thus taking intervals of one second and incrementing the cycle by 1.

curCycle++;

The time sensor loop is set to true to enable the sequence to stop and be restarted using the time sensor loop SFBool value.

kareokeTS.loop = TRUE;

A giant if/else value tests the cycle value (integer seconds) then sets the whichChoice values of the appropriate switch mapped in the script header. The script lets the music play for 9 seconds before showing the backdrops and lyric text.

This corresponds to the intro instrumental of the song, that is, the song title and album cover are displayed at 9 seconds, followed by the performing band name and a photo of the band at 13 seconds, followed by the songwriter name and a characture image, followed by the first line of the lyric and an image to illustrate that lyric at 23 seconds into the song.

if (curCycle == 0) { # 0 seconds
whichChoiceLyric = 0;
whichChoiceImage = 4;
// Blank
}

Note the background being unbound at this point. The flaming backdrop is active and supercedes the background. Unbinding the background sets the display behind the
backdrop to black. This takes care of occasional hiccups in the animation such that the background won’t peek out unexpectedly.

else if (curCycle == 9) {# 9 seconds
whichChoiceLyric = 1;
whichChoiceImage = 1;
background1.set_bind = FALSE;
// Taking My Time
}

Note the comments used to document what the current display is. This helps to keep up with the timing, the current displays, and the switch whichChoice value currently used.

else if (curCycle == 13) { # 13 seconds
whichChoiceLyric = 2;
whichChoiceImage = 2;
// Ground Level Sound
}
else if (curCycle == 18) {# 18 seconds
whichChoiceLyric = 3;
whichChoiceImage = 3;
// Songwriter: Len Bullard
}
else if (curCycle == 23) {# 23 seconds
whichChoiceLyric = 4;
whichChoiceImage = 9;
// My friend she cut me two lines
}


To show how the script can cause other effects, here is the set for the animation of the guitar solo.


else if (curCycle == 110) {
whichChoiceLyric = 27;
sweepcam0.set_bind = TRUE;
whichChoiceImage = 5;
background2.set_bind = TRUE;
mat.transparency = 1;
// solo - blank
}

At 110 seconds, the sweep cam is turned on by binding to it, the star background is
turned on, and the flaming backdrop transparency is set to fully transparent. The
foreground backdrop is set to the lightning animation. This is for the instrumental solo sequence.

else if (curCycle == 122) {
viewpointGen0.set_bind = TRUE;
// solo Blank
}

At 122 seconds, the static camera on the guitar player is bound while he plays his solo

else if (curCycle == 135) {
viewpointTaking.set_bind = TRUE;
}

At 135 seconds, the camera is moved back to the front main camera just prior to the
next verse lyrics and images starting

else if (curCycle == 137) {
mat.transparency = 0;
}

At 137 seconds, the flaming background reappears

else if (curCycle == 138) {
whichChoiceLyric = 28;
whichChoiceImage = 6;
// So don’t expect me to come
}

At 137 seconds, the next image and verse appears.


At the end of the sequence, the script does some housekeeping to reset the header fields for another play if the user decides to replay the sequence. The lyric display is set to the first element (zero) which is blank, the skull background is displayed, the record label logo is displayed, and the curCycle field is set back to 0.


if (curCycle == 215) {
background3.set_bind = TRUE;
kareokeTS.loop = FALSE;
whichChoiceLyric = 0;
whichChoiceImage = 4;
curCycle = 0;
}
}
]]>
</Script>


And that is all there is to it. This is really simple sequencing but given some clever uses of the different animatible objects, X3D can enable you to create serviceable and even edgy presentations of your music without the hassle and expense of making a video. Build it, link to it from your mySpace page and if you want a YouTube entry, use a frame grabber program to render it to a video format.

Bang! Done! Ain’t that cheap?

H-Anim Models

I am not documenting the musician and instrument models as avatar structure is not the topic of this tutorial. These are the property of Keith Victor and are H-anim standard avatars created with Flux Studio. The original models were created by Mauro Turri, aka, MrGB.

Their movement is created by the interaction of the AudioClip and AudioSensor values created when the mp3 audio file plays USEd by the Sensor in the AudioClip field. The clip is declared in a Sound element AudioClip subelement elsewhere in the world.

See below for basic documentation of the AudioSensor node and how it is set up to work in the demo. Note again that the AudioSensor is an extension node and not part of the X3D language standard. It is provided as-is in Flux Viewer installations.

The two X3D viewers I am currently developing for, Flux and Bit Management Contact, vary wildly in their display of H-Anim avatars. It is one of the known incompatibilities of these two implementations of the X3D language. Flux Studio can import a VRML97 Numedia Avatar Studio avatar that BS Contact plays correctly as output from Numedia as-is in the VRML97 output. Flux Viewer will correctly render the output of Flux Studio. Alas, Contact does not although the resulting rendering is good for startling impressionist effect, or just plain scary stuff. This is something the VRML/X3D authoring community is waiting for the two vendors to work out.

The Flux Audio Sensor

By code inspection, the following is what the sensor looks like and some guesses at what the values mean. Being an experimental node, it is also un-documented as far as I can determine. What follows is a description of the fields and their values as divined by inspecting the code in Keith Victor’s example at that Media Machines site.

Flux Audio Sensor

<AudioSensor DEF='AudioSensor1'
frequencyDataSize='16'
frequencyStart='0.05'
frequencyEnd='0.4' >
<AudioClip USE='AudioClip2' containerField='source'/>
</AudioSensor>

fields
frequencyDataSize='16'
frequencyStart='0.05'
frequencyEnd='0.4'

eventOuts
magnitude_changed SFFloat
beat_changed SFTime
frequency_changed MFFloat

AudioClip SFNode

By testing the node implemented only appears to work with mp3 files.

• frequencyDataSize: is this the bit sample size (eg, 16 bit vs 32 bit) and if so, does the implementation handle higher bit rates?


• frequencyStart and frequencyEnd: appear to be range values


• magnitude_changed Is this a peak magnitude (amplitude?), so corresponds roughly to the loudness relative to the overall sample or within some range?


• beat_changed: being a timestamp, this seems to be some form of sample calculation for metric beat?


• frequency_changed: an array of floats? A buffer?

The example provided by Keith in his band world uses a script to convert the events from the sensor in to values that control actions in the world using the usual orientation and position values that are routed from these nodes to the H-Anim avatar structures.