A brief rundown of data flow programming packages

I have been teaching myself yet another piece of data flow programming-media wrangling-presentation type software this week, and it has occurred to me just how many of these things there are now, and how completely baffling and overwhelming all these different packages might be.  Trying to figure out what does what and when to which one is pretty confusing, to say the least.  So, in an effort to summarize some of the packages available, I decided to do a blog post on the subject.

Derivative TouchDesigner is one I just started learning this week.  I am just barely getting my feet wet with it, but it’s quite amazing and intuitive.  From the TouchDesigner wiki:  

TouchDesigner is a software product from Derivative (Toronto and Los Angeles) which is used to build interactive 3D and 2D applications. It is “procedural”, “node-based”, real-time and is considered a visual programming language. It is designed to give its users enormous flexibility in building applications without needing to program in a conventional way.

When I first started watching the instructional videos on TouchDesigner, I thought the terms “CHOps” and “TOps” sounded awfully familiar, so it wasn’t much of a surprise to find out that its roots are in Prisms/Houdini.  

Here you can see a screenshot of the example project, and as you can see, it is definitely a “data flow” node-based approach to programming.  Scripting capability is available via Python (not sure where yet, but it’s there somewhere,) and you can access the 3D functions of your GPU, use OSC (Open Sound Control,) etc.  As TouchDesigner has its roots in graphics, both 3D and 2D, it makes sense that most content created with it is graphics-oriented.  I will make more entries about TouchDesigner as I learn it, because, as I mentioned, I just started with it.  But it seems like an enormously flexible tool.

Now onto our old standby, Max MSP.  I am much, much more familiar with Max, and while I most certainly can’t claim to be an expert in it, I have been using it since 2008 or so.  From wikipedia:

Max is a visual programming language for music and multimedia developed and maintained by San Francisco-based software company Cycling ‘74. During its 20-year history, it has been used by composers, performers, software designers, researchers, and artists for creating recordings, performances, and installations.

As you can see, Max also has a very data flow, drag-and-drop interface.  Scripting is available via javascript (which I have not used personally,) and you can get just about everything into and out of Max—OSC, Kinect, DMX, etc.  Max’s wonderful video extension, for both live video and manipulation of pre-rendered video, is called Jitter.  Max’s real strength is its documentation and its huge user base.  If you don’t know how to do it in Max, you can usually find out without too much trouble.  Max’s weakness, IMO, is when you try to go into 3D space.  My one experience with attempting to access openGL in Max consisted of printing out hundreds of pages of baffling instructions and tutorials, giving up, and then just coding everything in Processing instead.  Since TouchDesigner has its roots in 3D graphics, I have to wonder if it might be the ideal tool for that sort of work instead.

Alright, now onto the ones that I have absolutely no experience with, but they’re worth a mention.

VVVV is “a hybrid graphical/textual programming environment for easy prototyping and development. It is designed to facilitate the handling of large media environments with physical interfaces, real-time motion graphics, audio and video that can interact with many users simultaneously.”  As you can see, we have yet another very Max-like interface.

This is vvvv’s wikipedia entry.  

Pure Data is “enables musicians, visual artists, performers, researchers, and developers to create software graphically, without writing lines of code. Pd is used to process and generate sound, video, 2D/3D graphics, and interface sensors, input devices, and MIDI. “

As you can see, we once again have a very Max-like interface.  

Isadora is a package that’s enormously popular in theater and live performance.

"The award-winning, real-time media manipulation software to create stunningly interactive visuals, sounds, and environments."

And then there are more specialized entries such as Watchout and MadMapper for more timeline-based work and projection mapping specifically.  

There are also many, many others (like those specializing in VJ applications) that I haven’t mentioned, and then I suppose that one could argue that there is also some crossover with 3D content creation tools like Unity.

To complicate matters, there’s a lot of crossover among the tools.  While most people consider MadMapper to be a projection mapping tool, for instance, you can also do similar things with Max, TouchDesigner, or Isadora.  Right now I’m just going to focus on learning TouchDesigner and will blog about my progress here. 

Really awesome metal 3D printing via robotic arm

The MX3D-Metal is the latest project from Joris Laarman Lab.

Getting GIS data files into your 3D applications

It seems that GIS (Graphic Information Systems) are playing an increasingly important role in, well… everything these days.  At some point, you might need to import GIS data into your 3D application.  Let’s say you’re doing an architectural rendering, or a sight line study, or something, and you need to see how the terrain affects your site.  So, the typical method of doing this is to ask your GIS person for a 3D model of the terrain, and then bring that into your 3D application, merge your models… and then deal with a bajillion polygons eating up all of your RAM, crashing your computer, and taking up many hours of rendering time.

Guess what?  There’s an easier way.  Almost any 3D application these days (Maya, Cinema 4D, 3DS Max, etc.) will support Displacement Maps.  First of all, if you have GIS shape files (*.shx) you can view them in ArcGIS Explorer, which is a free download.  This should give you some idea of what your terrain is supposed to look like.

image

 At this point, most people ask their friendly GIS expert to export the terrain as some kind of 3D file, dxf, or possibly VRML, or fbx, resulting in a massively huge, lockup-inducing mesh of zillions of triangles and much frustration.  There’s another way!

Instead, ask your friendly GIS expert to export a DEM file.  What’s a DEM file?  It’s simply a grayscale raster image that represents the terrain instead.  And what are those useful for?  That’s right, displacement maps!  See, this is what our DEM file looks like:

image

Black represents the lowest points in the map.  White represents the highest points.  Now it’s a simple matter of placing one flat plane (it can be one polygon,) applying the dem file as a displacement map, setting your tiling and scale correctly, and you’ve replaced many, many triangles with one simple plane and a texture map.  This image took a scant few seconds to render in Cinema 4D:

image

You can then ask your friendly GIS person what elevations are the lowest and highest in the map, set your displacement map accordingly, and *poof* everything is to scale.  Remember that the GIS elevation is in feet above sea level!  This is why you have to ask for relative heights instead of absolute.

image

In this case, I have asked for a few reference points from the GIS data.  I have then eyeballed the locations on my ground plane in the XZ plane, then placed a cube and scooted it back and forth in the Y axis until it was sitting at the same height as the displaced terrain.  Then I took a note of the Y position in feet.

It turns out it is not too far off.  Of course, it’s not exact, and I did have to keep doing test renders over and over again to see where the placeholder cubes were, but it’s pretty close.  And again, using this method, the test renders are lightning fast.

There are limitations with this, however.  The displacement map typically won’t render in your working viewport, so you won’t get to see it until you render your project out.

image

So if your project requires very close work with the terrain (for example, showing a house sitting on a mountainside,) you’ll have to do things the old way, by importing 3D data directly.  An alternate solution is to “bake” your displacement map into your geometry if your 3D package allows you to do so.

The swallow-able video camera gains FDA approval

Pretty amazing technology: a video camera you can swallow that transmits video via wireless.http://lnkd.in/b9xpXM7 But, if you’ll remember, it was exactly this same technology that caused so much confusion for George Jetson, causing him to risk his life as a test pilot.http://lnkd.in/bcFNfgJ less