I am a UK based artist and technologist. As an artist, my work explores ways of opening up and challenging the nature of creativity itself, in particular through the innovative use of technology. Integrally related to my artistic practice, I work as a technologist and adviser with artists and organisations, specialising in free open-source technology. I am the developer of Praxis (http://code.google.com/p/praxis/), a modular framework for live creative play with video, images, audio, and other media. Praxis is built on the NetBeans platform. Neil C has posted 1 posts at DZone. View Full User Profile

Modular NetBeans RCP Framework for Video, Images, Audio, and Other Media

07.07.2011
| 7671 views |
  • submit to reddit

Praxis is a Java-based modular framework for working with video, images, audio and other media.  It has a particular emphasis on live creative play - everything should be configurable and connectable at runtime. Praxis LIVE is a graphical, patcher-style interface for developing Praxis projects 'on the fly'. 

Praxis has been developed by myself, UK artist and technologist Neil C Smith, and is available freely under the GPLv3 licence.  It has evolved from various bits of code I've written over the years in the creation of generative and interactive media installations, as well as live performance.  It is partly inspired by projects such as AudioMulch, Bidule and Isadora, and to a lesser extent Pure Data and Processing; however, it is not intended to be a clone of any of them.

Here is what it looks like, explanations and more screenshots at the end:

Primary Features


  • Media neutral architecture. Praxis is built around a simple but robust asynchronous message-passing system, allowing different media (and the system) to communicate with each other without interfering with each other's processing.

  • Intuitive patcher-style graphical editor. Drag and drop components from the palette on to the patch window; draw in audio, video and control connections; edit properties in the property window.

  • Edit everything live. Praxis and Praxis LIVE are built around the central concept that everything should be editable at run time.

  • Background loading of all resources. Load images, samples, etc, without interrupting playback.

  • Optimized, software graphics pipeline, including a range of pixel effects and blend modes. (A hardware-accelerated OpenGL version of this pipeline is in development).

  • Video input and playback using bindings to the GStreamer library. You must have GStreamer installed on your system for this to work. (Other video libraries may be supported in the future).

  • Low latency audio support. Through JavaSound and JACK.

  • Custom GUI's and MIDI control. Create custom controls panels bound to any parameter, or control any parameter using a MIDI controller. (A graphical editor for these is coming soon!)

  • And, finally, live Java coding. Praxis comes with a small but growing range of built in components, but for those times when the built in stuff doesn't do what you want, you can enter fragments of Java code, compiled and added to the graph while your project is running. The video code component provides a subset of the Processing (2D) API ported to the Praxis graphics pipeline, along with some additional features such as access to blend modes and animatable parameters.

Praxis and NetBeans RCP

Praxis has been developed using the NetBeans runtime container since fairly early on.  This has facilitated the modularisation of the code - one of my primary aims with opening up Praxis is not just to open up the project as a whole, but to open up various libraries I have developed for it as separate projects.  I'm aiming to get various audio libraries released separately in the near future, including the JNA based bindings to JACK.

Praxis has its own script syntax for defining and controlling patches - necessary because it abstracts out all the complexities of dealing with the asynchronous message system.  This syntax is quite simple and succinct, but a visual editor had always been an aim.  Development of Praxis LIVE started in April 2011 based on the NetBeans Platform, and a first binary release was made available in June.  This currently provides a visual graph editor for developing audio and video patches (based on a much adapted version of the VMD components in the visual library), though GUIs and MIDI bindings still need to be created using scripts.  By building on top of the NetBeans Platform, Praxis LIVE also benefits from the NetBeans project system, editor (with basic script syntax highlighting), properties window, palette and more.

Community Involvement

I have been a long-time user, advocate for and advisor on FLOSS software, but Praxis is the first project I have developed and I am still getting my head around the best ways of involving other people in it.  Development of the software so far has been very much a solo effort, and primarily driven by my own professional requirements (which after all, pays me to develop it!)  My primary focus over the next few months is promoting it to others who might have an interest in using it, from which hopefully I can build input into the project, be it feedback, suggestions, feature requests or, yes, code.  If you download it, please do sign up for the discussion group!

Recognising that Praxis is also a bit of a niche area of interest, I am also looking at ways of building some community interest in some of the libraries developed for Praxis - in particular I've been in contact with some of the other projects doing audio in Java and discussing ideas around code reuse.  I'm also interested in feeding into the NetBeans Platform community as my use of the platform develops.

Screenshots

A simple "hello world" Praxis style, showing the graphical patcher, properties system, hub manager, etc:

The same as above but in Praxis' script syntax (the graphical editor reads and writes using this same syntax):

A much more complicated video patch.  This shows some of the key additions to the VMD visual library components, including aligned ports and highlighted connections on rollover:

A live compiled particle system blended on top of a live webcam image.  Also shows some of the blend modes available in the underlying graphics library.  Syntax highlighting of Java fragments isn't enabled yet!

Patch showing two different audio patches running concurrently through Jack (you would normally connect some other audio software in between rather than just connecting Praxis to itself!).  This also shows a basic control panel example.

Published at DZone with permission of its author, Neil C Smith.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)