This is the SLMotion home page!

SLMotion is a tool (or a set of tools), primarily meant for video analysis of sign language videos. Of course, the tool may also be used for other purposes.

Releases

Release 0.1.0 is here:

• MacOS X bundle
• Source code

Beware of bugs. This release may not be very pretty, and it mainly consists of a simple dump of the repository. No clean-up of the code has been done. Please report any bugs (expected to be plentiful). A Ubuntu binary package is planned for realese in the near future.

Requirements

Slmotion is written in C++11. Hence, you will need a recent C++ compiler (GCC 4.6 or CLang 5.0 or newer is recommended). Also, the build process uses CMake, so CMake 2.8.0 or newer is highly recommended. Mac users need OS X 10.8 (Mountain Lion) or newer. There are also several library dependencies. Here is a list of some libraries that need to be installed before compiling slmotion:

• OpenCV 2.3.0 or newer
• LibXML2 (2.7 or newer recommended)
• Boost 1.36 or newer (with program_options, regex, system, filesystem, and python)
• BLAS and LAPACK (e.g. Netlib or Apple Accelerate implementations)
• Zlib
• Qt 4.8 (for the GUI)
• FFMPEG and swscale for video input (1.2 or newer recommended)
• Python (2.7 or newer recommended)

Doxygen should be installed for generating the documentation (where needed).

Also supported

• OpenNI for analysing Kinect recordings in the ONI format. (Currently only supporting version 1.5)
• Facial landmark detection using the flandmark library [1]

FAQ

Documentation

• User's guide
• Python API documentation
• Programming style guide
• Component reference

Issue tracking

Issue tracking will be issued at a later moment. For the time being, please report any bugs and other issues to Matti Karppa via email (firstname.lastname@aalto.fi).

Scripts

Here are some example scripts that you can have a look at to see how things are done:

• Visual silence detection: visualsilence.py and the corresponding annotation template visualsilence.ann
• Facial landmark detection: faciallandmarks.py. Detects facial landmarks in the image and stores the result in a feature file (CSV file). This script requires libflandmark.
• KLT/ASM hand/face tracking and statistical feature generation: kltasm.py. This script performs the actions specified in our 2011, 2012 papers (see the papers below). Warning: The current implementation of the ASM is somewhat broken. In addition, a lot of assumptions are made about the footage, so this should not be used for any serious analysis. It is here only to serve as an example of what can be done, but it is unlikely to work very well with arbitrary footage.

Licence

SLMotion is licensed under Simplified BSD Licence (2-clause BSD licence or the FreeBSD Licence). Please read the licence here.

If you use SLMotion while making a publication in an academic setting, please cite this work as follows:

  @InProceedings{LREC2014SLMOTION,
    author = {Matti Karppa and Ville Viitaniemi and Marcos Luzardo and Jorma Laaksonen and Tommi Jantunen},
    title = {{SLM}otion~--~An extensible sign language oriented video analysis tool},
    booktitle = {Proceedings of 9th Language Resources and Evaluation Conference (LREC 2014)},
    publisher = {European Language Resources Association},
    year = {2014},
    address = {Reykjav\'ik, Iceland},
    month = {May}
  }

Papers using SLMotion

• Marcos Luzardo & Matti Karppa & Jorma Laaksonen & Tommi Jantunen. Head Pose Estimation for Sign Language Video. In Proceedings of the 18th Scandinavian Conference on Image Analysis (SCIA 2013), Espoo, Finland, June 2013. Available online at http://users.ics.aalto.fi/jmkarppa/scia2013headpose.pdf.
• Ville Viitaniemi, Matti Karppa, Jorma Laaksonen, and Tommi Jantunen. Detecting Hand-Head Occlusions in Sign Language Video. In Proceedings of the 18th Scandinavian Conference on Image Analysis (SCIA 2013), Espoo, Finland, June 2013. Available online at http://users.ics.aalto.fi/jmkarppa/scia2013occlusion.pdf.
• Matti Karppa, Tommi Jantunen, Ville Viitaniemi, Jorma Laaksonen, Birgitta Burger, and Danny De Weerdt. Comparing computer vision analysis of signed language video with motion capture recordings. In Proceedings of 8th Language Resources and Evaluation Conference (LREC 2012), Istanbul, Turkey, May 2012. Available online at http://www.lrec-conf.org/proceedings/lrec2012/pdf/321_Paper.pdf.
• Matti Karppa, Tommi Jantunen, Markus Koskela, Jorma Laaksonen, and Ville Viitaniemi. Method for visualisation and analysis of hand and head movements in sign language video. In C. Kirchhof, Z. Malisz, and P. Wagner, editors, Proceedings of the 2nd Gesture and Speech in Interaction conference (GESPIN 2011), Bielefeld, Germany, 2011. Available online at http://coral2.spectrum.uni-bielefeld.de/gespin2011/final/Jantunen.pdf.

Contact

The chief maintainer of the program is Matti Karppa. Any inquiries regarding the program can be sent to firstname.lastname@aalto.fi.

References

[1] M. Uřičář, V. Franc and V. Hlaváč, Detector of Facial Landmarks Learned by the Structured Output SVM, VISAPP '12: Proceedings of the 7th International Conference on Computer Vision Theory and Applications, 2012. pdf