From Dokeos

Video Conference Client

Build Notes

see VideoConference_BuildNotes

Installation

see Installing_videoconference

Main Features

• 2-4 Users Cam+Audio
• 1 Chat
• 1 (shared) Whiteboard

Description

With the VideoChat-Conference Tool people should be able to connect to the App and see themselves Video/Audio. They have also the possibility to send textmessages to a seperate Chat-Channel. Besides of that they have a whiteboard where they can draw something, write text, dragNdrop some images on the whiteboard. The content of the whiteboard should be possible to download as a compiled SWF. Users can upload Files (but only images can be used through the whiteboard). To make the whiteboard downloadable as an swf it will be compiled by the LaszloServer.

Technical Background

http://www.osflash.org/red5

http://www.openlaszlo.org

http://www.openlaszlo.org/Video

The OpenLaszlo Application will be integrated into Dokeos. This Application will use the Video-Components which are available in the nightly build of OpenLaszlo (Version 3.4.x) (nightly build) or in Version 4.x

The Red5 to be used for (deployed)development is Version 0.5 (0.6 rc1 date is 16 October) : http://www.webbase-design.de:5080

Locally I'm also testing with the 0.6-release from the trunk directory.

The current version needs *both* LaszloServer AND Red5.

The OpenLaszlo (LaszloServer) part will later be removed when the videoconference application will be ready to be delivered pre-compiled.

Currently Red5 is based on Jetty 6.0 Servlet Container, which is just an embeddable Web server written in Java.

OpenLaszlo is based on the Tomcat 5.0.30 Servlet Container.

The OpenLaszlo Servlet is currently incompatible with Jetty 6.0 (cause it uses commons-http-client-2.0 while Jetty uses 3.0) and it is not tested with Jetty 6.0. The next OpenLaszlo version (Project Name *Legals* is expected to be released in december 2006).

TODO

• check if everyone in classroom mode receive the stream of everyone without even asking for it...
• document how poll works
• delete extra files from Openlaszlo subversion
• import red5 source code into subversion

CSS design

The new version of the videoconference tool (to be out at the same time as Dokeos 1.8.5) can be theme-modified via CSS. Here are some tips and tricks to do this, as it is not completely using the CSS standards, but is largely inspired by them. You will need the videoconference tool sources (downloadable from SVN) in order to modify the style, and then you will need to recompile the application with OpenLaszlo 4, as the CSS is statically bound.

Tips and tricks

• A CSS can be loaded from the main .lzx file by using the following syntax, right after the <canvas> tag opening:

 <stylesheet src="resources/css/videoconf.css" />

• The CSS cannot dictate the canvas dimensions. These have to be defined in the <canvas> tag attributes width and height.
• When using a CSS, the property to be used has to be bound to the object by defining the object in t}e CSS with either:
  • in the case of an object having a specific tagname:

 objectname{
   property: xx;
 }

• • in the case of having an id attribute in the object:

 #objectid{
   property: xx;
 }

• • in the case of having a name attribute in the object:

 [name='objectname']{
   property: xx;
 }

• Underscores (_) cannot be used in object names or ids or name attributes because this will cause OpenLaszlo to break upon compilation
• If you want do define a 'visible' attribute via CSS, you have to define this attribute last in the tag declaration, and you have to use a value of 0 or 1 (you cannot use true or false). The CSS property used could be 'display' (that sounds the closest to normal CSS)
• Property values should be expressed without units (34 and not 34px). They will be considered as pixels.
• If you want to load a specific resource (external image or something similar) through CSS, use the 'source' attribute in the OpenLaszlo object declaration, not the 'resource' attribute, which will want to load a previously-declared resource.
• In the CSS, embedding an id tag (#someId) inside a name tag ([name='someName']) seem to block the effect of the id tag styling. In fact, it seems that all kind of hierarchy is blocking the normal behavior of the CSS (this is in contradiction of the OL documentation). The id tag being unique as a basic prerequisite, you can safely ignore the name tag hierarchy prefix.
  • This doesn't work

 [name='someName'] #someId{
   height: 80;
 }

• • This does work

 #someId{
   height: 80;
 }

• If you define a class in OL with a tag <class ... > and a 'name="className"' attribute, then this name is considered as a tag, and as such, your CSS should define its style only by using the name of the class, like shown below. If you try to enclose className into a name tag in CSS ([name='className']), it will not work (your style will not be found, which will trigger an OL warning)

 className{
   height: 20;
 }

Design Screen

http://www.dokeos.com/forum/viewtopic.php?t=6633#28208

WhiteBoard

See WhiteBoard-Page: Whiteboard

Current Version

Authentication

Dokeos is a PHP/MySQL based Application. The SessionId of the User is stored in the database. The CLient gets the SessionId from the Cookie and will check it by making a query to the database. The Client will connect through Red5 to the Database. So there is a need in the Database for at least a new host to the current MySQL user running Dokeos (cause PHP will use localhost cause in most cases it is running on the same server) but Red5 could be on a seperate Server (and will probably often be) so there is a need for another MySQL user. Red5 is based on the Spring Framework. The Client will use Red5 and Hibernate to connect to the Database.

Server architecture

You can get a quick idea of the server architecture by having a look at the following diagram which represents a typical communication between a web client (browser) and the Dokeos servers.

What you can pick-up from this schema is:

1. a browser requesting a PHP page from the server will trigger processing by the PHP5 module of Apache
2. the response is sent as HTML
3. a browser requesting media files will simply trigger a hard-disk recovery of the file by Apache on the server
4. the response is sent as media files
5. when the user starts the videoconference, he does so from a link in his browser, and the request is transferred from Apache to Tomcat, which uses its OpenLaszlo module to translate the videoconference scripts into a Flash applet
6. the Flash applet is returned to the browser (and acts as a mini-browser itself for most functionalities)
7. the Flash applet opens a connection to the Red5 streaming server and communicates directly with it for the videoconference traffic

Development/debugging documentation

This doc is written as an attempt to find ways to better debug/develop on the current videoconference tool (which still has a lot of space to develop).

Warning: The code for the videoconference was originally written by a developer with 
 unperfect knowledge of English. This has led to a series of badly named variables 
 and class names. Although we are doing some efforts to fix this, you might fall upon
 some disturbing names. If you really need to get the meaning of a variable, first try
 re-ordering the letters, then try reading it with German roots. That's how we do it...

The current development version is now inside the subversion of Dokeos. You can reach the web interface to the SVN sources here: http://dokeos.svn.sourceforge.net/viewvc/dokeos/trunk/videoconference/

Helper tools

As described very late in the OpenLaszlo documentation, you can use the Eclipse XMLBuddy extension to help a little in the XML structure edition while developing with OpenLazlo.

OpenLaszlo offers a very nice and easy to use Debug tool, which is enabled by setting the debug attribute to true in the <canvas> object. This, however, cannot be used in production environments as it shows the debug window for everybody.

If you know how to call the videoconference tool directly (with a pre-set URL) you can also activate the debug window just for you by appending the debug=true parameter to the url.

Short Glossary

Although a glossary generally comes at the end of the document, understanding the following terms might considerably help you in reading the following documentation.

• rtmp: Real-Time Messaging Protocol (http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol)
• rtmpt: Real-Time Messaging Protocol Tunneling
• OpenLaszlo: A language made from a mix of XML and JavaScript, that generates Flash or AJAX applications
• canvas: The root element of most OpenLaszlo script files (if you don't know what an XML Root is, you should read documentation about XML first)
• library: The root element of most OpenLaszlo included files

Structure

When you install the videoconference tool following the documentation, you should have something like this kind of path:

/home/dokeos/lps-latest/

where you can find the following subdir:

/home/dokeos/lps-latest/Server/lps-latest/dokeos/videoconference/

This is where most of the files for the graphical interface (Flash) can be found.

Dokeos uses two files: videoconference.lzx and videoconference2.lzx.

It uses videoconference.html and videoconference2.html, which both call the corresponding lzx versions. This might change in the future so that Dokeos can call the .lzx files (or they pre-compiled versions) directly from the Dokeos file hierarchy (which will give us the opportunity to pass some variables straight to the generated Flash applet rather than getting them from a call to checksession.php afterwards).

An LZX file is a big XML definition for a class that is then interpreted by the OpenLaszlo server to generate a Flash applet (see link to the OpenLaszlo website above).

The initial LZX file (in this case videoconference*.lzx) contains the canvas (the equivalent of an <html> tag for an HTML document).

The canvas generally contains include statements to include all the other classes, and several handlers to handle global events. Each class generally contains handlers as well (oninit, ondata, ...) which will trigger along the use of the videoconference tool, governing its process.

Finding your way inside the code (for newbies)

A good way for looking into the code for newbies (while we are still lacking documentation) is to track the pictures used.

The different pictures/icons that are used for the final graphical interface are defined in the resources/library.lzx file.

From there, you can trace where they are used by looking in the other files for their object names (e.g. xoffice22_rsc).

We find that one of the pictures is used in conference/tabbar/menubardokeos.lzx.

In the menubardokeos.lzx file, we find the definition of the different buttons of the interface. We're interested in the action triggered by the upload button here, which is called uploadtabutton, so we trace this one down but get nowhere, because the menubardokeos.lzx file is used as a complete object somewhere else (in videoconference.lzx).

Nevermind, in the uploadtabutton definition, we find that we instanciate an uploadtab object, so let's trace this one down.

We find this class is referenced in a whole bunch of scripts, but the definition (including the <class name="uploadtab" string) is located in conference/tabcontent/uploadtab.lzx so off we go, etc.

Have a look at the following visual map of the application's interface. You will find that most sections of the interface have a description telling you what file it comes from.

Getting info from checksession.php

The Flash applet, upon initiation, uses the Apache context of the user session to call a script in the Dokeos hierarchy called main/webrooms/checksession.php.

This script is parsed inside the getSessionVarsData method in dokeos/videoconference/conference/global/functions.lzx

The structure of the XML results sent by checksession.php is as follows:

<?xml version="1.0" encoding="UTF-8" ?>
<dokeosobject>
 <courseobject>
   <id>01</id>
   <name>XML : Lenguage de programacion</name>
   <official_code>01</official_code>
   <sysCode>01</sysCode>
   <path>01</path>
   <dbName>dokeos_01</dbName>
   <dbNameGlu>dokeos_01`.`</dbNameGlu>
   <titular>Julia Villa</titular>
   <language>spanish</language>
   <extLink>Array</extLink>
   <categoryCode>PC</categoryCode>
   <categoryName>PC Skills</categoryName>
   <visibility>2</visibility>
   <subscribe_allowed>1</subscribe_allowed>
   <unubscribe_allowed>1</unubscribe_allowed>
 </courseobject>
 <userobject>
   <user_id>4</user_id>
   <firstName>James</firstName>
   <lastName>Heny</lastName>
   <mail>email@address.domain</mail>
   <lastLogin>1185477990</lastLogin>
   <official_code></official_code>
   <picture_uri></picture_uri>
   <language>spanish</language>
 </userobject>
 <library>
   <attribute name="rmpthostlocal" value="my.dokeos.net" type="string" />
   <attribute name="iswebrtmp" value="false" type="boolean" />
   <attribute name="rmptport" value="1935" type="string" />
   <attribute name="rmptTunnelport" value="8088" type="string" />
 </library>
</dokeosobject>

So, blaming the lack of XPath usage in functions.lzx, we can see that to get access to an element like the user's official code, we end up with this line in functions.lzx:

 canvas.setAttribute('official_code',value.childNodes[0].childNodes[1].childNodes[5].childNodes[0].data);

If we split it, we get:

• value = the XML document
• childNodes[0] = the root (<dokeosobject>)
• childNodes[0].childNodes[1] = the <userobject> element
• childNodes[0].childNodes[1].childNodes[5] = the <official_code> element
• childNodes[0].childNodes[1].childNodes[5].childNodes[0] = the content of the <official_code> element
• childNodes[0].childNodes[1].childNodes[5].childNodes[0].data = the data inside the content of the <official_code> element

It may seem like this is much more hassle than is needed... (no "but"s here)

checksession.php contains now:

• user informations (name, firstname, login, status (if he is a moderator/a student)
• the current session id of the user (Flash do not use cookies of the user's browser)
• the current course / lp / slide (for the audio recorder)
• the ip/port adress of the red5 server

All these informations are read by functions.php as said before and put inside 'attribute' of OpenLaszlo.

The flash application take one parameter: dokeosPath. In order to pass this parameter to the application, you should use the param html tag if you are inside an HTML page or call add it inside the url. For example: flash.swf?param1=value1. DokeosPath contains the url of the dokeos portal. For example, if you use this url: http://myLaszloHost/conf.swf?dokeosPath=myDokeosHost the flash application will try to download checksession.php from http://myDokeosHost/main/webrooms/checksession.php

If this parameter is not provided to the flash application, it will looks for checksession.php inside the same directory. So https://myDokeosHost/main/webrooms/videoconf.swf will download checksession.php from https://myDokeosHost/main/webrooms/checksession.php.

Link with Dokeos: api.php

This file is included with Dokeos 1.8.3. This script allows the new version of the video conference application (to be released at the same time as Dokeos 1.8.4) to read/list/delete files from documents inside the current course of the user.

Starting from Dokeos 1.8.3, only with the development version of the new video conference application (see Installing videoconference) the application use Dokeos PHP code in order to upload presentation files and convert them to images. This now allows the moderator of the conference, for example, to import a slide from a learning path onto the whiteboard.

The path to api.php is detected like the path to checksession.php, using the calling URL of the videoconference applet.

api.php is called, mainly, from remote/mainservice.lzx

Getting language vars from the XML language files

The language vars handling is not the best thing in its current state. The main problems in the current state are:

• that the language files are not used everywhere
• that the files are statically put in the videoconference code directory, which means it is not bound to the DLTT tool (i.e. it cannot be translated as other normal language vars in Dokeos)
• that a arbitrary list of message IDs is used, which makes it impossible to get what a message might be from analysing the code (you have to search for the message ID in the language file at the same time
• that the possible language files are hardcoded in the code, making it difficult to extend to new languages

So let's at least describe the structure and the way it is loaded. You can find the language files in dokeos/videoconference/. There are three files (for each of the three languages supported): english.xml, german.xml, french.xml

If we open one of these files, we will find something like this:

<language>
       <labelfield>
               <labelid>1</labelid>
               <labelvalue>Caméra</labelvalue>
       </labelfield>
       <labelfield>
               <labelid>2</labelid>
               <labelvalue>Micro</labelvalue>
       </labelfield>
       <labelfield>
               <labelid>3</labelid>
               <labelvalue>Nom</labelvalue>
       </labelfield>
       <labelfield>
               <labelid>4</labelid>
               <labelvalue>Effacer le commentaire</labelvalue>
       </labelfield>
       <labelfield>
               <labelid>5</labelid>
               <labelvalue>Progrès</labelvalue>
       </labelfield>
       ...
</language>

Such a file is parsed and integrated into the parseLanugageObject method (note the spelling mistake), which is called by the languageData dataset definition (correct spelling here).

Basically, the parseLanguageObject method puts everything in an array of type t2[labelid] = labelvalue and then pushes the resulting array into the setLabelObject() method.

A possible very big improvement for this method would be to use a wrapper script that would prepare the variables coming from Dokeos into an XML format for the videoconference tool to later come and get these variables. That would allow us to continue using the DLTT from Dokeos to improve the videoconference translation.

How language vars are used internally

The new videoconference tool (being coded as of Nov. 29th 2007) introduces a label-based identification for translations, which means the XML file looks like this:

<languageobject>
       <labelfield>
               <labelid>Camera</labelid>
               <labelvalue>Caméra</labelvalue>
       </labelfield>
       ...
</languageobject>

This means we can use Dokeos language files (through parsing to XML by checksession.php) and so the DLTT tool to translate the videoconference tool interface.

Translations are then put (in functions.lzx::i18ninit()) into a global object called _labelObject. This object is used by the canvas.translate() method (also defined in functions.lzx).

The translate() method can be called directly (see YouAreNotModeratingThisCourse label in videoconference.lzx) or via the labelid property of some objects, like labeltooltip, which have a setter method defined which calls canvas.translate(). Suffice to set the labelid property of the labeltooltip object to a translated label id to get it automatically translated.

Connecting to the streaming server

Funnily enough, the connection to the server is not located in anything that makes sense, but rather in a script called loginscreen.lzx and burried deep into dokeos/videoconference/conference/popups/. The connection consists of a call to rtc.connect();, rtc being the Remote T? Connection.

The difference between the speakers and the spectators

The conference room mode gives a good idea of what the difference is, in effect, between the speaker and the spectator. The speaker has his picture shown and can speak while others listen. The sectators can only see and listen.

However, there might be cases where we want the spectators to be able to talk, for example, for a short period of time (say, for a one-minute question to the speaker). There is an ongoing development to get this feature inside the Dokeos videoconference soon.

But how does it work? Upon opening of a new connection, the Red5 streaming server takes parameters which tell if the newly connected user will have the right to speak (i.e. if the streaming server needs to listen to the voice and video flow from that user) or not. When we want to alter this, however, there is only one almost-elegant way of doing it: cutting and reinitialising the user connection.

The feature we want to get in the end is that the spectator should be able to speak and show his video flow until a limited time has gone past or until the speaker/moderator decides to cut him. Upon the first of these events happening, the spectator stream has to be cut and reinitialised with different parameters.

The Flash 9 player and Linux Webcam problem

Various items on the Adobe website give us explanations about problems of webcams with the Linux Flash 9 player:

http://weblogs.macromedia.com/emmy/archives/2007/01/adobe_flash_pla_1.cfm Excerpt:

What is the work around for getting a web-cam to work with Flash Player 9 for Linux? It has been stated on the Adobe website that only Video4Linux v.1 is supported.

However, v.1 was deprecated in June of 2006 and now all Linux kernels are running v.2 with compatibility API for v.1. We have not been able to get Flash Player 9 for Linux to recognize a web-cam using the v.1 compatibility that v.2 of video4Linux has with it.

Has Adobe verified a driver configuration for that allows web-cams to be utilized with Flash Player 9 for Linux in lieu of the deprecated v4Linux v.1?
Posted by: Chad Halvorson at June 4, 2007 01:43 PM

The same page gives many other comments regarding this specific configuration.

The weblog of the Flash-for-Linux developer (yes, there is only one) is here: http://blogs.adobe.com/penguin.swf/2007/06/fullscreen_beta.html

Link between RED5 and the flash application

Generic Documentation

Call a RED5 function from Openlaszlo

Into Openlaszlo

example:

       <netremotecall name="getClientListScope" funcname="getClientListScope" remotecontext="$once{ canvas.thisrct }" >
           <handler name="ondata" args="value">
               <![CDATA[
               //JS Code
               ]]>
           </handler>
       </netremotecall>

name attribute is the name of the method into openlaszlo

funcname is the name of the function into red5

remotecontext is an established link between openlaszlo and red5

ondata handler is called when the application received data. Each call is asynchrone.

Into RED5

every public method defined into Application object can be called from OpenLaszlo

Call an Openlaszlo function from RED5

Into RED5

There is already a private method inside the Application object which send a message to all client but the current one:

   private void sendClientBroadcastNotifications(IBroadcastStream stream,String clientFunction, RoomClient rc);

stream and rc can be defined using RED5 internal data. For example:

   sendClientBroadcastNotifications(stream,"closeStream",ClientList.get(Red5.getConnectionLocal().getClient().getId()));

Call "closeStream" function on all clients but the current one with rc (which contains informations about the sender, instance of RoomClient)

Without sendCleintBroadcastNotifications:

   ((IServiceCapableConnection) conn).invoke("funcname", new Object[] { param1, param2, param3, ... }, clientId);

Into Openlaszlo

       <netremotecall name="closeStream" funcname="closeStream">
           <handler name="ondata" args="value">
           </handler>   
       </netremotecall>  

value will take the value of rc. You can get the value from one attribute of rc using the folowing syntax:

   value.name_of_the_attribute

RED5 Dokeos Application

How Dokeos/Openlaszlo Streams

(For this section, we describe how videoconference works. The way the recorder does is nearly the same.)

Openlaszlo streams using the Flash api directly. So in order to get more documentation, you should go on [1]

Search inside "Client-Side ActionScript Language Reference for Flash Media Server 2", "Client-Side ActionScript Language Reference"

There you will find documentation about class named "Camera", "Microphone", "NetConnection", "NetStream" and "Video".

The NetConnection is instanced under the name canvas._nc into conference/rtmpconnection.lzx. It's started during application initialization.

Camera and Microphone seems to be defined inside conference/video*/editrecordstream.lzx (see below for more informations about this file)

The interesting part: NetStream is instanced and started into videoconference/video*/editrecordstream.lzx, into the method "doinitalize" which is also called during the application initialization. It takes as parameter reference to the Camera and Microphone object. The name of the instance is cancas._ns. This function also call this :

  canvas._ns.publish(canvas.streamid,"live");

This method, according to Adobe's Documentation, starts to stream Camera and Microphone in "live" mode. If you want to record the stream on the server (like the recorder does) you have to use the "record" mode.

streamid is set by the RED5 server. When Openlaszlo connect the red5 server, this one use the following line:

  service.invoke("setId", new Object[] { conn.getClient().getId() },this);

In order to return to the client his id.

In conference/videoconnection.lzx, there is the implementation of setId which only set the streamid.

So, When Openlaszlo start, it immediately start to stream everything to the server. He is identified with a streamId he received from the server during the connection. During the connection, he also send everything he knows about the user (see RoomClient class, RED5 application, in order to get the list of attribute sent to red5).

Differences between MeetingRoom and ClassRoom

Between MeetingRoom and ClassRoom everything but two folders is the same.

The first one includes conference/video/* while the second one includes conference/videoconference/*.

In theses two folders, there is the same number of files but some of them differ.

videoobject.lzx defines the graphical widget which will be used to show the webcam video of only one user. The only difference between the two implementations is the size of the objects.

videocontainer.lzx defines all the top left panels with one or more users on it. It also implement a method: startStream (see How Openlaszlo handles a new user).

library.lzx is the same files in both folders.

editrecordstream.lzx is the same file but the names of some objects vary.

How Openlaszlo handles a new user

When somebody calls the NetStream.publish method, RED5 will call the newStream function on each client of the same room.

This function is implemented on the Openlaszlo side info conference/videoconnection.lzx.

This function will pass the information to the function startStream which is implemented inside conference/video*/videocontainer.lzx.

At present time it is not known from where comes the userpos attribute but It seems to contains the position of the user in meeting mode (from 0 to 3) or 0 in class mode.

In both case, this function seems to check if the position is reserved for this user... If it passes, it connects the streamid to the flash video component and starts to play it.

In meeting mode, there is a function addItem which is called after receiving data after a call to getListClientScope into videocontainer.lzx, but at the present time I do not understand where this function is called...

Other resources

A project of documentation in spanish has also been started by Miguel and vastly improved by Julia here: Video Conferencia