Well the visuals are coming from projectors. From the picture, they're mounted somewhere "behind" the camera. You can see them projecting onto the dancer on the top right.
If you look just on the left side of the picture, they're a guy with a lap. It looks a lot like Resolume, which has visual library built-in and you can download to make effects.
This actually did happen today lol, I’m controlling the color of a resolume background through grandma2. I made a generic rgb fixture and then mapped the rgb colors to the rgb colors of the colorize effect in resolume.
I followed this guide to get things working, https://resolume.com/support/en/connect-grandma2
we didn’t change any network adapter properties, just messed with the universe and network outs in ma2 and the the receiving universe in resolume. Both computers are hardwired into the same network.
We had been trying for a long time to get this to work and finally got it with the help of a friend. The resolume background is essentially it’s only lighting fixture that I have control of, dimmer and everything
There are a number of possible approaches. If the music is 'locked down' then the visuals could be edited to the music and simply played back. It is more likely that a number of sequences were created and played back using VJ software (I mostly use Resolume ). This would allow the playback of the sequences to be controlled in real-time. This could be matching the playback rate to the tempo or a video effect that is controlled by the sound of the kick drum or an almost limitless combination of control sources and parameters to be controlled. It is feasible that the images could be generated in real-time in response to the music but I think this is less likely. It could be a combination of all of those approaches, layered on top of each other.
Resolume Arena is what I've used in the past. Not sure of "best" as that's subjective to workflow and I haven't touched it in a while. It does also have a free trial version that I think displays a watermark but will let you mess around with the software.
That's only for the mapping portion. The content still has to be created. That's through photoshop+aftereffects or blender or Maya or some other animation workflows. Look at r/vjing for more info from people who actively do this.
This is absolutely not how timecode-synced visuals work with Rekordbox & Resolume
CDJs output a timecode which makes it trivial for the VJ to sync their visual clips; artists like Tipper have been doing this for decades
Resolume is free to download (it throws a watermark on the screen every minute or so until you buy a license, but it's a minor nuisance if you're just messing around), and also has really solid free training on the website ( https://resolume.com/training ). I highly recommend starting there. I have very little background with any kind of visual work, but a month ago I downloaded Resolume, started grinding through the training, taking notes, and practicing everything I learned, and I've already gotten to the point where I'm making stuff that I'm really proud of. From what I've seen from a month of obsessively reading through posts on here, pretty much everyone uses Resolume.
https://resolume.com/download/
Scroll down to the bottom and find the version of 5 you want from the dropdown box. Email Resolume about the key, if you're already a customer they should take care of you.
Looks like resolume is coming up with a solution but it caught em off guard and it's going to take a bit
http://resolume.com/forum/viewtopic.php?f=12&t=15950
https://resolume.com/support/en/no-dxv-export-in-adobe-cc-2018
Source: I wrote a plugin for concert visualizers to rotate equirectangular videos. I use this for displaying and rotating 360 videos in a planetarium dome and other immersive environments. If you want to play with it yourself, download a trial of Resolume or any other FFGL-enabled software and install my plugins in the plugins directory.
I usually ask the audio technician (if there is one, otherwise the venue is probably so small I know the DJ personally, so I ask him/her) to plug me in the audio mixer where there's usually a monitoring line that's free.
You'll probably need something like this:
Laptop 3.5mm Jack to Mixer 2 x 6.3mm Mono Cable 2m
If you need more than 2 meters than you should try looking for a longer version or buy this thing with a longer 3.5" to 3.5" cable separately:
3.5mm Stereo Jack Socket to 2 x 6.35mm Mono Jack Plugs
Again, this is something the audio guys maybe already have covered but it never hurts to bring some of your own gear :)
Additionally, if you're interested in some of the higher-level stuff I suggest looking into OSC signaling - but that one requires a lot more cooperation from the audio producing side, i.e. the DJ.
EDIT: The eBay suggestions were made considering you have a 3.5" input in your laptop. If you're using another audio device with your VJ setup (since you're new to this I'm thinking no :P) your results may vary.
Here you go:
https://resolume.com/manual/en/r4/controlling#audio_analysis
So you first need to plug in your audio-in line (or onboard mic would do, too) and then choose 'External' - just check your Preferences so that the correct audio device is recognized. I find that you sometimes have to relaunch Resolume after changing the audio driver option, but other than that it's really quite simple.
I never said what you think I said. But he definitely has a VJ team. Almost everyone uses prerendered content. You have too. Ex just does more of the work. There's still laser and lighting people.
You can read the interview with his VJs for more background.
https://resolume.com/blog/14554/examining-the-executioner-excision
It's most likely a bootstrapped (or a 3rd party) plugin for Resolume. https://resolume.com/software/
source: I'm a python coder and run grassroots festivals in Canada.
Sometimes I'm using this one but it needs to be fixed as for some buttons do not work for newer versions of Arena (OSCTouch Editor on mac is great piece of software for that - as well for building your own layout)
https://resolume.com/forum/viewtopic.php?t=13504
E: forgot the link to forum...
What exactly do you need PBP to do for you? Because resolume performs best when you use Alley to convert your content anyway, so codecs shouldn't really be relevant to any Resolume projects. If resolume Alley won't convert for you, process it thru Handbrake first. Then it should work.
Aside from that, you'll be dealing with possibly as much as a quarter to half second delay from the extra processing time to receive Playback Pro's content and re-send it from Resolume. Whatever audio you've got would certainly be out of sync.
So anyway, to answer your question, yes. You CAN recieve playback pro in Resolume. Blackmagic makes some "cheap" Video capture cards that Resolume Arena 6 supports. I've used it to apply special effects to camera sources before being output to screens. click this link and choose the "capture" category.
Macbook pro 15 inch with a GPU? Is your content converted to DXV?
All of your content needs to be in DXV so it can run through your computers descrete GPU. If you don't use DXV resolume will be unstable, and if you don't have a GPU you can only run a layer or 2 at most anyway.
If you read the section of the manual on this topic and related forum posts (old but still relevant), you'll see that the record function isn't really designed to record whole shows. Further the disk operations related to it have a low priority to reduce impact on the actual output, so this may cause frame stutter if you're recording to the same disk where your files are stored.
One way around this is to use a syphon/spout output to a second program (or even computer) which is dedicated to recording the content/show.
One other popular way would be to use an external recording device, as discussed here many of these can be set up independent of any computer and record to an internal drive, reducing I/O on the drive your content is on.
Sadly none of this helps your current predicament, where you've already captured the show, but hopefully you can avoid this in the future.
I believe the record functionality is one of the things high up on their fix list for V6 onward.
I'm not familiar with how Resolume handles MIDI output, but I've played around with getting QLC+ to talk to my APC40 mk1. From skimming the communications protocol for the mk2 it looks pretty similar. Each pad responds to a series of "notes", and depending on which note you send it, it will change different colors. The mk2 is a bit more complicated because it has RGB LEDs in the pads, but the gist of it is the same. You can use a program like Bomes MIDI Translator to detect incoming MIDI messages (like Pad #2 hit once) and if-then out the corresponding messages to turn the LEDs on (like Pad #2 turn green).
Edit: looks like Resolume supports MIDI feedback out of the box, so no need for Bomes
HAP codec is probably the most used format among VJs for not bogging down resources as much as say PhotoJPEG. However, Resolume software, one of the more popular programs, does not support HAP. Instead they developed their own codec called DXV. The current version is DXV3.
You can download HAP codec here: http://vdmx.vidvox.net/blog/hap
You can download DXV3 here: https://resolume.com/software/codec
It would be nice if we could just get a more uniform dimension size, codec, frame rate, naming scheme for VJ clips.
This. OSC is the better way to do it. Resolume has predetermined OSC message addresses - unlike apps like VDMX which have learn modes. Resolume's manual has some info on it's OSC implementation: https://resolume.com/manual/en/r4/controlling#open_sound_control_osc
This video I put up a few days ago may be usefull if you haven't dealt with OSC in max before: https://www.youtube.com/watch?v=bupVHvMEAz0
Synesthesia is a great program indeed, lots of possibilities and jaming I use it in combination with Resolume Arena
If I should make something the crowd could interact with, I would use RestAPI to setup a website the crowd could goto via smartphone and mess around with parameters or change clips.
​
Its actually not so difficult: https://resolume.com/support/restapi
The difficult here lies in the open-gl render path.
This means that the gpu us used wich renders the ui. Intel optimus is a crosspoint router for display streams. And an dedicated gpu game as an example is routed through the intel gpu matrix.
But the edid is negotiatet by the intel chip wich is the feature open-gl hooks on.
For reaolume there is an workaround crating an fake game profile...
https://resolume.com/support/en/optimus-workaround
Simpliy click run with nvidea wont help. I tried and mostly get crahses then. Using an external display is mostly the hardbreach method.
You’re not as far off as I think you think you are. The hardest thing is that your laptop probably can’t support four independent outputs, so you’d need to look into a datapath or maybe a second (synced) laptop
Here is a good write up on your options:
https://resolume.com/support/en/lots-of-outputs
Resolume arena 6 is a great option.
In terms of projector brightness — if you just want to do a proof of concept, borrow a bunch of cheap old classroom style projectors. Literally anything will work in a room with total darkness.
Use a projection throw calculator online (lots of results on Google, they should all work) where you can plug in the model number, lens, desired screen size, and see how bright your image will be.
If you’re willing to share the general budget range you expect to spend on this (<$1,000, $10,000, $50,000 would all be totally different recommendations) that would help.
Also is this a permanent install or something you’d want to tour with? Does it need to run unsupervised for weeks on end like a museum or can you dedicate someone to operate it like a show?
If you don't need the outputs it's always better to have only one card in your system. Resolume only renders on the card that is displaying the interface window (so never plug your monitor into your on board graphics output), and pushing pixel data between cards takes time.
Unless you have a very good reason to use them (i.e. frame lock / gen lock), I would ditch the workstation components. A current gen i9 with a current or last gen graphics card (2080, 3080) will outperform your current system by at least 2-3 times. Get 32 or even 64 gigs of memory and you have yourself a beast. Of course the same is true for the AMD/Radeon equivalent.
You didn't ask about many outputs but this is a very good read nonetheless:
https://resolume.com/support/en/lots-of-outputs
There's a nice writeup from the resolume folks on this:
https://resolume.com/support/en/lots-of-outputs
Although it's getting old and doesn't mention NDI.
Oh .. and PS .. it handles OSC out via scripting .. so the next show we're adding video and I am gonna use it to control Resolume Arena ( if i can find the $$ ), Mitti and/or Open Broadcast Suite .. again very flexible if you're into linux/python, etc.
This was an event in Dubai, the artist was a fairly big DJ (Sven Vath).
The diamond projection screen elements were requested in the client brief. My wife made the screens using PVC plumbing tube and screen fabric. It came out pretty awesome.
Visuals were generated by a program called Resolume for those interested.
I also previously designed the stage, trussing, audio and lighting in sketchup, which is why I had the detailed model. We had to measure the entire site with a laser as there were no existing CAD files available prior. It was a bit tricky and time consuming, but once I had the model, it made doing event designs like this MUCH easier.
There is an issue with openGL rendering and nvidia optimus. (switch between low poer intel in geforce card)
the programm seeks for the graphiccard hosting the output connector wich is the the intel.
Nvidea renders the game and passes the pixel strem to intel wich shows them on a display..
most games can be forced to run on nvidea. But special programs like obs & rsolume need to be in charge of the output port (e.g framerate setting) and then are hooked to the intel.
​
https://resolume.com/support/en/optimus-workaround
try this with obs instead.
Steps to have a video influence the Milkdrop visuals
>I see, you're actually receiving effect data from that Resolume service via Artnet protocol, reduce it for a limited number of LED, and feeding the strip. Very cool project, man, very cool. And how many LEDs are per stripe?
Yup. This is the tool resolume uses to dither the animations and create artnet packets.
And thank you! It was a lot of work, so I'm glad you and others appreciate the results.
>First of all thanks to you, Scott, and all the other maintainers of this project. You guys made a really great tool, even if you’re not actively working on it right now.
Yup! I used Resolume Arena to generate effects, run FFT for audio visualization, and unicast artnet (dmx) packets to each esp.
I use this Artnet client on the esp to receive packets and integrate it with Adafruit Neopixel (the fastest library for large #s of ws2812bs).
Steps to have a video influence the Milkdrop visuals
https://resolume.com/forum/viewtopic.php?t=12345
A few suggestions here - noting this one in particular -
Turns out Windows 10 automaticly sets 2nd screen on 75hz, but when i set it on 60hz the problem was solved.
Are you sure you have tried using an .wav file? Mp3's aren't working as far as I know.
So you need to convert that file, or even better ask the DJ for an uncompressed .wav file to make it work in Resolume.
You can read more about that at this link.https://resolume.com/support/en/preparing-media#audio-compression
yes, he is using a VJ on this tour. they are like DJs in that they have a stash of clips (prerendered video and real time graphics) that they synchronize to the performance in real time using manual or automated methods (like a music visualizer). they blend all that stuff into the video feed, in this case the green screen, and blast it to the large screen.
https://resolume.com/software is a popular VR software if you're more curious.
There's so much you can do with Resolume.
This thread on the Res forum has dozens of examples of skilled users creating visuals using no source clips, only the built-in effects engine.
You can download each user's composition and deconstruct how they did it.
You're welcome. If you have access to a macbook, I would recommend thinking another way about this. https://resolume.com/software Resolume Avenue for example. Beat matching is automatic, and you can live record your edits. It's a LOT more fun and possibly more-appropriate for your usecase.
A software like Resolume (https://resolume.com/) would do the trick. They have a lot of preloaded graphics that you can edit on the fly with different effects. It pretty easy to learn and even easier to operate. You can even set up some faders on your light board to control it. It definitely would be my go-to for something like this.
Resolume has a free program that lets you convert image and video formats into DXV3 and you can further customize the settings such as alpha channels, sizing, audio (if it has any), etc. Here's the link and you can find it towards the bottom of the page "Resolume Alley" https://resolume.com/download/
> Ah, so like a technical rider then. What are the questions you'd consider mandatory, especially when it comes to the larger places?
Sort of, but more the conversation you have with the person/organisation booking you.
The obvious things, like:
etc...
Basically anything you'll want from them or they'll want from you should be sorted before you turn up.
I'm going to guess that any freeze or skips are a cpu or gpu performance issue on your Resolume computer. I assume that you're generating SMPTE from a stable external device and that there's not a skip or freeze problem with the SMPTE input.
You should probably make sure that you're running at optimum performance before synching to SMPTE, and make sure that you don't have a fundamental problem. This might help: https://resolume.com/support/en/preparing-media
Once you get the system running optimally, then see if you're having a sync issue. Make sure you've read this page and that you're sure you've chosen the right SMPTE framerate: https://resolume.com/support/en/smpte
Maybe not *exactly* what you're looking for, but I would surely recommend looking into Resolume. It's free to try and has loads of effects and possibilities for live video mixing. There is also a lot of automation possible.
Another option is using the Winamp visualisations. They are automated and you can use your audio as input for the video. There are a lot of good visualisations out there. The customisation options are limited however.
You can use resolume for that. Out of the box it works with midi input data, you just need to assign midi to the parameters you want. Routing the midi data from logic to resolume shouldn’t be to difficult, at least it isn’t from Ableton. https://resolume.com/support/en/midi-shortcuts
Thanks for that. There's a lot of that in the industry. Can't say it doesn't hurt, but I have lots of xanax. I understand that lots of people don't like breakcore, and don't fault them for it. This guy was out of line though, and I've literally never seen him contribute to this sub before.
As for OSC, it's a relatively simple network protocol that's addressed similar to HTTP or SOAP/REST APIs. You can see all the resolume addresses by going to mapping and selecting OSC. It's similar to mapping midi controls, except instead of training resolume to read the control, you're training the control to talk to the addresss.
​
Here's resolume's guide: https://resolume.com/support/en/osc
Yes, usually there's a lighting person/VJ who does visuals, using software such as Resolume
Interestingly enough you can also use a MIDI controller for controlling lights, the same as you can for DJing. Technically you could use CDJs for controlling lights and visuals. Usually they'll use something a bit more suitable though, rather than decks. They don't need to know the songs that will be played and can do it on the fly usually, same as how you can mix on the fly without pre-planning your set.
This is easy to do and there are many different ways you can do it -- you could assign a midi button to the bypass button of that effect, you could map a fader/knob to the opacity of that effect, or you could put the effect directly into the composition and trigger that slot.
​
The official Resolume training has good training on using a MIDI controller -- scroll down to 6.2 and 6.3: https://resolume.com/training
Actually, maybe pass on the TripleHead. It was the DataPath I was thinking of. Resolume's documentation talks a little bit about it here. It basically acts as a monitor with a large output resolution to your GPU, and you can configure the DataPath itself to split that large output into smaller chunks. I've pushed data through two of them at a club I used to work at, but never configured one myself so I don't know all the details.
What resolution projectors? The main thing you're going to need to keep in mind is the maximum resolution of the graphics card you go with. For example, the max on the RTX2080Ti is 7680x4320, which means theoretically you could run 16 displays from it at 1080p. I've never attempted that, so there are probably some technical issues you would run into with that, be it performance or something else.
The other thing you would need to consider of course, is outputs. Most likely your best bet there would be to use something like a Matrox TripleHead. I would avoid multiple graphics cards if possible, IIRC there can sometimes be syncing issues and Resolume (which I assume you're using) has some limitations with multiple GPUs.
Resolume has much more MIDI capability than what’s credited here. It can assign MIDI values to almost anything, including variable slider values with mapped ranges. Ranges can honor note velocities. Every note/cc/channel combination can be specifically mapped and controlled and ranged, and each map can be saved/swapped out based on anything, including MIDI triggers themselves, allowing for endless flexibility in how the MIDI signals are applied to controls.
Many drum machines are capable of sending MIDI on more than a single channel, often with different instruments (parts of the kit) on different channels. At a minimum, some software in front of Resolume could re-channel any output from the drum machines. That’s not going to be the limitation here. It’s certainly much less limited than fft response which can only isolate freq ranges, and can’t separate a kick from a low tom, for instance.
I don’t know what you’re referring to as “gate” in the MIDI spec (maybe you mean CV gate?) but there isn’t a gate in the core MIDI standard.
There’s a good overview of Resolume’s MIDI mapping capability here: https://resolume.com/support/en/midi-shortcuts
I know it gets said on every post, and I know it's expensive - but I'm gonna be that guy to recommend Resolume.
You have the hardware, and a residency. I don't see any reason why you shouldn't be bringing an A-game performance, or at least developing one over time.
A slowly growing (but growing!) community over at /r/VJloops would love to help you find materials to work with. Once in Resolume you can boost those visuals into a new dimension worth of unique creative output, though I would also flag that garbage in; garbage out. AKA, you've still got to supply creative artistry rather than just mashing the effects. But do mash the effects, we all had to get it out of our system at the start.
Best of luck! There are a heap of materials on Reddit and the internet to help you get started with Resolume. Once you start really getting in the groove, make sure to share one of your shows with us!
resolume is what a lot of my VJ friends use https://resolume.com/
you'll have to find/create content somewhere else, but simple videos and even still pics can be really effective if you get some good effect chains going. set it up with a midi controller and its a lot like DJing in ableton's session mode
You can just run an aux cable from the audio out on one to the audio in on the other.
As for video going back, it's a little more difficult.
If you have a video capture device, you could send video back over an HDMI port. Look for streaming capture devices designed for consoles, like the mirabox. I don't know how much delay you'll get with that, though. When you introduce hardware that does processing on that video, the delay will vary based on the size of the video.
Spout and NDI are network video protocols. Their delay will be higher than HDMI. (HDMI delay is 3-5ms, where network protocols could be as high as 50ms or as low as 10ms.) However, since I don't know what kind of delay a mirabox or other video capture device will introduce, It might be 20ms delay, or it might be 100ms delay, I don't know if you will get better performance from one over the other.
Luckily, as far as video hardware goes, a stream capture device is relatively cheap. You can pick one up for less than 150 dollars.
NDI and Spout are built into resolume automatically. https://resolume.com/support/en/syphonspout
Apparently, there is a plugin for OBS that adds NDI support: https://obsproject.com/forum/resources/obs-ndi-newtek-ndi%E2%84%A2-integration-into-obs-studio.528/ Had no idea and that's super useful to know.
Based on this new information, I recommend you try using NDI first. Spout is the most compatible and expedient, but NDI usually has better performance. Since you can test this for free, it's a no loss scenario. If the delay is too high, get back to me. I've got a few networking tricks, and we can look at some capture cards to see if we can get better results.
I use Processing (which is a visual programming language) the most.
I currently use a Dell XPS 13 from 2015 and I find that while some stuff runs fine at 60fps, if i have too many objects in the stuff I write in processing then framerates can drop right down. I also cannot screen record what I'm doing because it wrecks the frame rates (although I think using a hardware solution for recording my display out might be a better option anyway)
I also want to use openframeworks which is similar to Processing. Resolume which is a VJing software. Edit video in Adobe Premiere (my current laptop Premiere crashes often). Maybe use Unity.
This benchmark guide for Resolume might be helpful? https://resolume.com/blog/11093/performance-benchmarks
I find the results spreadsheet completely overwhelming because of all the different things. Maybe you're able to parse it better?
a computer with multiple outputs and cables of equal length, GPUs will output frames at the same time and the sync is fine. Even when running 2 standard GPUs the sync will be pretty much fine, however with installs and high end clients you would use Datapath FX4s(4k to 4x 1080p outputs) or Nvidia Quadros with a sync card to make sure your outputs are perfectly in sync when you need tons of outputs.
The specs on that will work great. Here is Resolume's recommended specs.
Edit: just realized I skipped over the content creation part. Yes, it will be great for that as well.
Software: Resolume Arena v6.0.7
OS: Windows 10
Link: https://resolume.com/download/
Description: Is a tool for live VJ-ing.
Reason: I am a start-up freelancer and i don't have the money to buy'it. The v5 is very limited, and the v6.0.1 that it is already cracked is full of bugs and crashes in less of 5min. it is unusable
Source: I wrote a plugin for concert visualizers to rotate equirectangular videos. I use this for displaying and rotating 360 videos in a planetarium dome and other immersive environments. If you want to play with it yourself, download a trial of Resolume or any other FFGL-enabled software and install my plugins in the plugins directory.
Are you looking at a way to create live visuals? AE isn't designed to be a real-time application. You would do much better with some VJ software. I use Resolume but there are plenty of other options.
You might want to head over to /r/vjing
if you are interested in doing motion graphics for live electronic shows check out the software resolume as well as beeple.
if you don't end up doing live shows - the things you learn in the process will definitely position you for a job in the future doing stuff you like (interactive installations / projection mapping / motion design.)
Photoshop won't help you here, you'll need to step up your game to a real video editing software (or adobe media encoder, perhaps). Resolume uses an uncommon video codec which you'll need to download from somewhere. Check out its manual or FAQ to find out which one you'll need.
EDIT: This one
Resolume VJ software with audio reactive video clips. If you're going to use Resolume, remember to convert all your video clips to the DXV codec
Korg nanoKontrol midi controller
Projector looks to be at least 2000 lumens or brighter