Igevorse Otonnoleare

Category: Google Summer of Code 2014 · Feed


Using MuseScore 2.0 with VST instruments

Hi!

In previous post I wrote a list of features I've implemented in MuseScore this summer. Now I made a demo of using MuseScore with VST instruments via JACK:

Yngwie Malmsteen - Far Beyond The Sun

Connections & Tracks

Connections in patchbay (clickable):

  1. Lead Guitar

    I used Suburban Guitarist for synthesizing clean guitar sound and then added a bit of distortion with Guitar Rig 5. Used preset: Solo Maker.

  2. Rhythm Guitar

    DSK Guitars Acoustic produced nice sound for the rhythm. Guitar Rig 5 with "Boosted Solo Metal" preset made dirty sound from that.

  3. Bass Guitar

    I used simple DSK BassZ plugin.

  4. Church Organ

    I've pre-recorded an organ staff using Church Organ 2nd VST(i) to demonstrate you using JACK Transport. I loaded this track into Ardour 3, and as you can see it started playing with MuseScore at the same time.

  5. Drums

    Superior drummer is a well-known plugin for the musicians. I used Rock Solid preset.

VSTs are runned by FeSTige, final mixing is done with jack_mixer.


Software

MuseScore 2.0

jack2 1.9.10

Ardour 3

Catia 0.8.1

FeSTige 1.0.2

I recorded this video with SimpleScreenRecorder and edited with Kdenlive.


Used VST(i)

Suburban Guitarist

DSK Guitars Acoustic

DSK BassZ

Church Organ 2nd

Guitar Rig 5

Superior Drummer

Do you like it?

· Retweet Retweet! · Share Share · Comments (1) ·

Project summary

Hi!

I want to say thanks to my mentor Nicolas Froment (lasconic) and MuseScore community for the help and answering questions. Thank you for your contributing and evolving MuseScore, it is a great project! Also thanks to Google, due to GSoC I made my first contributing steps in the world of opensource.

It was a long journey to the world of code, it was very interesting and fascinating to dive into unknown code, to make a research in unknown areas, to invent new approaches and to create something new. New code, new features, new possibilities and new happy users. Sometimes new bugs too Laughing.

I hope my work was useful for the community, developers, users and the future of the project.

What I've done

Here is a list of features I've implemented while participating the GSoC 2014:

  1. JACK MIDI Out support (PR)

    This feature gives an ability to send MIDI events to one or several external synthesizers or VST plugins.

    Profits:

    • More realistic sound of score.
    • Adding realtime midi effects to the sound.
  2. JACK Transport (reposition) (PR)

    We provide current position in frames (samples) to JACK Transport, keep an eye on it and seek if position changed.

    Profits:

    • Seeking to some position in MuseScore causes seeking in all applications (audio/video players, DAWs, etc).
    • Seeking in any application connected to JACK causes MuseScore to seek.
  3. JACK Timebase Master (commit)

    Being the Timebase Master application could provide bar (measure), beat and tempo information to JACK.

    Profits:

    • Sync other applications working with midi to MuseScore, managing tempo.
    • More control over JACK Transport (not just seek, but slow down/accelerate).
  4. JACK Timebase Slave (commit)

    As a Timebase Slave we can read information about bar, beat and tempo and sync to any application connected to JACK as a Timebase Master.

    Profits:

    • More control over syncing to another application.
    • Adjusting tempo in real time to application / external automation tracks.
  5. Changing JACK preferences without restart (commit)

    This feature gives an ability to enable/disable JACK parameters like "Use JACK MIDI Out", "use JACK Audio" or "Become a Timebase Master" without restarting an application.

    Profits:

    • No need to restart MuseScore if you want change preferences related to JACK.
  6. Restoring JACK connections (commit)

    Now we can save audio and midi connections to other applications on exit or changing preferences and restore it when you run MuseScore again.

    Profits:

    • You don't need to connect all things again manually.
  7. Automatic control over JACK/ALSA MIDI Out ports (Upd Nov, 29: merged) (PR)

    This feature allows MuseScore to determine how many JACK MIDI and ALSA MIDI ports it should have and automatically add/remove ports when needed.

    Profits:

    • No need to manually control port number.
    • One more step to implementing ALSA MIDI.
  8. Assigning channels to instruments (not merged) (PR)

    With this feature users can assign any midi port and channel to instruments.

    Profits:

    • Channel information wouldn't be lost if you add new instrument, reorder staves or change midi program.
    • Better control over midi subsystem (you can even assign different ports for each instrument).
    • Ability to "compress" number of channels by assigning the same channel for a several, for example, piano staves.
    • Enhanced abilities to import and export scores (implemented import channels from *.ove, *.gtp*, import/export to *.mcsx, *.mscz, *.midi, *.mxl).
  9. Improved Instument Change element (not merged) (PR)

    Several Instrument Change elements with one midi program are "joined" together and shown as a single item in mixer.

    Profits:

    • No excess items in mixer (particularly important for big and complex scores).

Also, there were a lot of bug fixes, related and not related to JACK, improvements of UI, code optimisations and other things.

What's next

I have to test the last two features in the list and fix the bugs. Also, I promised in my blog to improve MIDI Actions, so I have to implement it too. The bad news is that I have a little time: my next university semester starts at September, but I'll try to implement these features in my free time.

But anyway feel free to ask me about my work or write me about bugs in IRC igevorse or via email: igevorse /at/ gmail.com

Regards,

igevorse

· Retweet Retweet! · Share Share · Comments (0) ·

'Assigning midi channels' feature is ready for testing

Overview of work this week

In short:

  • Implemented improved InstrumentChange element.
  • Implemented import midi ports/channels from Guitar Pro and Overture files.
  • Fixed bugs with assigning midi channels to instruments.
  • Fixed other bugs.

This week I finished improving InstrumentChange element. If you remember, the goal was to show only one item in mixer for the several same InstrumentChange elements. Now it's implemented completely. You can see an example score on the screenshot: we have several InstrumentChanges for Flute, but only one item in mixer. InstrumentChanges are synced, so if you change volume for "Flute" it would affect all Flute InstrumentChanges within the staff (within the Part, if we talk about code).

Also I was working on assigning midi channels to instruments. The first implementation was buggy, so I had to fix more than 10 quite serious bugs: each of them could break the feature. It works now, but solution is complex and affects a lot of processes (loading/saving file, adding/removing new elements, working with sequencer, etc.), so I am afraid I could broke something. There is only one way to be sure: test it hard.

Users was waiting for this feature for years, so if you're reading this post and you can help with testing - you're welcome! My pull request is here. Feel free to clone and test it!

As I said, current implementation is complex: partially because I added an ability to "fill the holes" in midi mapping. Suppose you have several instruments with midi channels 1, 3, 4, 5, 13 and the ports equal to 1. Now if you add a new instrument, it's channel would be mapped to midi channel 2. In previous implementation it would be mapped to channel 14.

Some information for the developers:

I mentioned in previous post that instruments could have the same midi port and channel, but could not have same indexes in midi mapping array. Now they could. It is the simplest approach to "join" all InstrumentChange elements to one item in mixer. All InstrumentChanges with the same midi program just mapped to one channel. Unfortunately, this solution has it's own drawbacks. So, a few InstrumentChanges with different instruments could be showed as one instrument in mixer. If they have the same midi programs, of course. For example, "Flute", "Soprano flute" and other related to flute instruments have the same midi program. This is probably a limitation in a GM soundfont standart. Or I just need to use not only midi program, but instrument->trackName() too for identifying an instrument. I am sure there's a user that needs to make an InstrumentChange from Flute to Soprano flute, so sooner or later I'll have to deal with this. If you have any suggestions - please contact me at any time.

Since now we support assigning midi ports and channels, we can extend our possibilities in import. I implemented import ports/channels from Guitar Pro 3, 4, 5, 6 (*.gtp*) and import channels from Overture format (*.ove). The last one does not support assigning midi ports to staves, as I understand, but supports assigning midi channels. Also, I have no information about *.mgu and *.sgu formats - there is no trial version of Band-in-the-Box application for the research. If I am wrong, feel free to write me.

Also this week I fixed two bugs: issues #29776 and # 29841.

Get ready for the final report!

Key tasks that stalled

It's ok.

Upcoming tasks this week

GSoC is almost over, but I promised to implement MIDI Actions, so I'll find the time for implementing. However, it may take a long time because of beginning of the study in university.


You can find me on IRC #musescore as igevorse or write a comment under the post.

· Retweet Retweet! · Share Share · Comments (3) ·

Fixing bugs is also important

Overview of work this week

In short:

  • Fixed bugs with assigning and syncing midi channels

My mentor found a few bugs in my implementation of midi channels, I was fixing them this week.

The first "bug" is actually an unimplemented feature: instruments are still visible in mixer after deleting. I didn't tested it and forgot about implementation. Actually this bug is more serious and leads to another bugs.

Here's some info for developers:

Every instrument has a pointer to the instance of class Channel. We have an integer there that shows our channel number. So, instr->channel->channel may have a value 0, 1, 2, etc. But it is not a midi channel (remember, we have to keep midi port too). We have a midi mapping (Score::_midiMapping[]) to save midi ports and channels. Our instr->channel->channel is an index in this array.

The main function here is Score::rebuildMidiMapping(). This function clears midi mapping and fills it with new values for each instrument: [port 0, channel 0], [port 0, channel 1], etc.

In my implementation I can't just clear it and assign channels because user can change midi channels. When user changes midi port or channel, we change the port and channel directly in _midiMapping. But when user creates or deletes an instrument we should add or remove a midi mapping. It's not easy to find all places where I should change midi mapping directly. Also it might be too difficult to a new developers. So, I avoided direct changing of _midiMapping. Also, since we don't clear our midi mappings, the order of items in mixer wouldn't be changed if we change it in the Add->Instrument window. I fixed it too. The new implementation of rebuildMidiMapping() works like this:

1. We pass through the _midiMapping array and search for the mapping to instrument that does not exists, then delete it.

2. We pass through the Instruments and search for the instrument that does no have a midi mapping (then it's a new instrument) and create a midi mapping for it.

3. Reorder mappings to the order in Add->Instruments.

Now it works pretty well.

The second bug was related to syncing channels in mixer. Since now we can have 2 or more instruments mapped to one midi port/channel, these instruments can't have different program, volume, pan, etc. When you change the volume of one of these instruments, volume of another instrument with the same channel would be changed too. There were some bugs but now they're fixed.

Also I accidentally fixed a few bugs with mixer and midi export.

Key tasks that stalled

Fixing bugs slows the development process :(.

Upcoming tasks this week

Again:

Check JACK implementation for possible bugs.

Finish improving Instrument Change element.

Implement import ports and channels from gtp*, mgu and ove formats (not sure last two support it).


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

Let's improve Instrument Change and add MIDI Actions

Overview of work this week

In short:

  • Implemented automatic control over ALSA ports number.
  • Implemented import/export of musicXML files.
  • Started working on MIDI Actions and improving Instrument Change element behavior.
  • Bugs fixed.

I wanted to become working on JACK MIDI In this week, but was surprised: it was already working!

I finished implementing import and export port/channel information to musicXML format. Since MuseScore does not fully support this file format, my implementation is not full too. But never fear: it works well. As my mentor said, musicXML is a large standard and there is no full implementation of it in any score writer. Also I've done some research and realized that bww, md and cap file formats do not allow saving ports and channels (write me if I am wrong).

Even if we don't have ALSA MIDI Out working, we should keep in mind it. That's why I changed part of code that refers to registering ports and added a support of automatic port creation if MuseScore need it (I made a similar work for JACK ports last week).

Thanks to pedroseq for his comments, he gave me ideas about implementing new features. In first, there is an idea about improving Instrument Change Element. Now when we add this element to staff it creates a new instrument and adds an item to mixer. If we have several instrument changes from Flute to Piano, we'll get a lot of different items in mixer for the same instruments (Flute, Piano, Flute, Piano ...). The idea was to combine same instruments to have only two items in the mixer: Flute and Piano. Described behavior (not combine) maybe considered as a feature: you can set volume, pan, chorus and reverb separately for each instrument. But we decided that new behavior might be better: Staff-text and System-text elements already combine channels, let's do the same with Instrument Change. My implementation is almost done: channels combine successfully, but I have to solve some side problems: for example, "mute" and "solo" options do not work properly now.

Another idea is adding MIDI Actions to all instruments (now we can send MIDI Actions only for Aeolus). MIDI Actions would give us an ability to send MIDI Actions to internal and external synthesizer at some point of time. It is not about music notating, but it affects the sound of score: you can add a portamento, then remove after 5 measures, you can change pan/chorus/reverb, change release time or modulation settings. Since internal synthesizer does not support all midi messages, it's better to use an external one. MIDI Actions have a great potential and implementing it can lead to a big changes in MuseScore. Maybe it will ever have an automation tracks for the midi parameters: volume, pan, chorus...

Key tasks that stalled

All is well.

Upcoming tasks this week

Check JACK implementation for possible bugs.

Finish implementation of MIDI Actions (although I'm not sure I'll finish it in one week).

Finish improving Instrument Change element.

Implement import ports and channels from gtp*, mgu and ove formats (not sure last two support it).


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

Assigning channels to instruments: save and restore

Overview of work this week

In short:

  • Implementing automatic control over JACK ports number.
  • Implemented import and export of ports/channels in midi and mscx/z files.
  • Fixed bugs with selecting audio system and sending "program change" midi message.

This week I was implementing automatic control of JACK MIDI Out port number. Before we had a spin box in Preferences->I/O that helps to specify number of ports, but we didn't have an ability to select ports for each instruments. All instruments were mapped to port 1. Now we can assign channels and ports to instruments and now we do not need a special controls for specifying port number - MuseScore already knows it! If you assigned 4 ports for your instruments, you'll immediately get 4 JACK MIDI Out ports.

Assigning channels is good, but saving them is better. You can save channels in MuseScore native formats (mscx, mscz) and MIDI format (mid, midi). Next week I would make import/export of ports/channels for other supported formats.

There is one unimplemented, but very important feature in JACK MIDI Out: it doesn't send "program change" MIDI message and if you play via external synthesizer, all your staves would sound as grand piano. It is fixed now: changing "sound" in your mixer changes program/patch in the external synthesizer. Of course, if you have the same soundfonts.

Also I've fixed the bug with selecting default sound system on linux. If you run "mscore -F" without pulseaudio installed, no audio system would be checked by default and you'll get a crash in some cases. Now it is fixed.

Key tasks that stalled

It's ok.

Upcoming tasks this week

Finish implementing import/export port/channel information to MusicXML, gtp*, bww, ove and other supported file types.

Begin work on JACK MIDI In.


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

Assigning channels to instruments: first steps

Overview of work this week

In short:

  • I began implementing assigning channels to instruments.
  • Found bugs related to changing instruments.
  • Fixed bugs with JACK Transport.

I decided not to implement "assigning channels for JACK MIDI Out", but began implementing more complex "assigning channels to instruments" feature. Users have been waiting for this, and now I can "kill 2 birds with one stone" - implementing this feature gives me proper midi channels for JACK MIDI Out too.

So, I continued researching of code related to MIDI and begin implementing.

At the beginning I wanted to change a lot of code to make internal midi system more simple. I changed some code and got partially working solution, but stalled with further implementation. After additional research I realized that current implementation of midi is convenient enough and I can implement this feature with minimal changes of code. My second implementation was simplier.

We can't change channels in "Create Instruments" window because there are only Parts and Staves, not Instruments. So, now we can change port/channel directly in Mixer window.

Here you can see the screenshot of new Mixer window:

It is already working solution, not a sketch. I tested it with QSynth (it shows the input channels) - channels really change.

Now I should implement saving channels to score file and restoring it.

While implementing "assigning channels to instruments" I was researching an existing midi-related code and found a few bugs with "Instrument change" and "Staff-text". Some of them can be fixed, but some not. It's more a design problem.

For example, user can create a piano staff, then add "Instrument change" and change instrument to violin from the 2nd measure. Then he can add "Staff-text" to 4th measure to make violin sound like a pizzicato violin. Now, what should we do if he deletes "Instrument change"? Should we get a piano staff with pizzicato violin from 4th measure? Or piano staff only? It is more a design problem.

Also my mentor found a few bugs in my JACK Transport implementation, I've already fixed them.

Key tasks that stalled

It's ok.

Upcoming tasks this week

Continue implementing assigning channels to instruments: save/restore channels to score file.

Check channels in midi export.


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (5) ·

Bugs & magic with cross-thread function calls

Overview of work this week

In short:

  • I was fixing bugs.

Yes, it is very simple. I just fixed bugs.

I found these bugs (8-12) while working on JACK Transport.

I did not work with count in or tempo slider in play panel, but I've implemented JACK reposition, and these features used my new code.

But I spent a lot of time on the second-to-last bug (11 on the "Bugs" page). It seems like very simple: I was getting a crash when I press "Play" on JACK Transport and there is time 0:00. I've checked all my code that implements reposition from JACK Transport and found the cause: uninitialized iterator Seq::guiPos. MuseScore crashes when was touching it in HeartBeatTimeout() - this function updates the cursor on the screen.

This iterator is initialized explicitly in seek() and we calling seek() when start playing. So, what's the problem? The problem was - I think you already got it - in my implementation. Seq::start() does not always call seek() because of some conditions I wrote. The solution is quite simple: I should change or roll back conditions. But in this case I will get JACK Transport reposition non-working.

Finally, I got it to work, but caught an another bug, very strange one. That's where the magic begins. In some cases MuseScore crashed with different errors about memory management, sometimes canvas didn't repaint. Actually, I spent most of the time on fixing these two bugs.

When I realized that I was stuck I wrote to my mentor and he gave me an idea: I did not use seek() function correctly. Really, I didn't think about threads a lot and used seek() wherever I wanted. Seek() calls functions that should be called only from GUI thread, but I called seek() from realtime thread.

So, the solution is splitting seek() into two functions: the first should be called from the GUI thread, and the second - from the realtime thread. Now we have seek() and seekRT().

After fixing the last bug I've checked all new features again and made a pull request.

Key tasks that stalled

Cross-thread function calls. I wrote above.

Upcoming tasks this week

Implement "Assigning channels to staves" feature I described in first posts.

Hot switching drivers is partially working, so there is no need to hurry up with it.


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

New UI: improved usability

Overview of work this week

In short:

  • Implemented JACK Timebase slave.
  • Implemented hot changing parameters of JACK.
  • Changed Preferences->I/O UI.
  • Implemented remembering/restoring JACK audio connections.
  • Fixed bug with repeats. Found new bugs.

I've implemented feature requested by daeavelwyn: JACK Timebase slave. In short, this feature allows MuseScore to sync to JACK's tempo. Now you can change tempo in application (current Timebase Master) connected to JACK and MuseScore will change internal tempo too. Of course, I am talking about relative tempo now, it applies only to playback tempo and doesn't change tempo marks in the score.

Anytime you can explicitly make MuseScore a Timebase Master and control tempo from MuseScore. This is possible now because of the second implemented feature: changing parameters of JACK driver without unloading it and restarting program. Now you can open Preferences->I/O, change preferences in "JACK audio server" section, click "Apply" or "Ok" and continue work on your score. In general, it gives you an ability to:

  1. Enable/disable JACK Audio and JACK Midi.
  2. Change number of midi ports dynamically (increase, decrease).
  3. Become a Timebase Master or not (new checkbox "Timebase Master").
  4. Enable JACK Transport or be independent (new checkbox "Use JACK Transport").
  5. Turn on/off remembering connections.

As you understand, I had to slightly change the Preferences->I/O window. Now user can not select more than one driver.

Also I've implemented remembering/restoring JACK audio connections. It was already partially implemented, MuseScore remembered connections not from patchbay, but connections made in Preferences->I/O. I removed extra controls and now MuseScore remembers connections you made in patchbay. There is no limit to number of ports. Also, if you disable "Use JACK Audio", but stay enabled "Remember connection(s)", the next time you enable JACK Audio, ports would be auto-connected. The same thing with MIDI ports.

I fixed bug with repeats mentioned in previous post, but found new. The first bug related to "count in" feature: JACK transport begin playing in the same time with counts, but should begin after count in ends.

The second bug related to Play panel and tempo slider. When changing tempo with this slider, MuseScore keeps current position in uticks, but not in samples (number of current sample changes, because it related to the tempo). So, I got seeking forward when increasing tempo, and seek backward when decreasing it.

But never fear - these errors not in master branch, only in my code. However, as a new features too... All described features are in jack-transport branch now, I'll make a pull request only after fixing all bugs.

Key tasks that stalled

All is well.

Upcoming tasks this week

Fix new bugs.

Fix old bugs (marked notes, crash on start - mentioned in previous post).

Think more about hot switching audio drivers.


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

JACK Transport: code published

Overview of work this week

In short:

  • JACK Transport reposition: code published.
  • Found two bugs with repeats.
  • New ideas about Timebase slave.
  • Ideas about hot switching of audio drivers.

As i wrote in previous post, my code of JACK Transport reposition worked, but wasn't ready for merging and could break working with other sound-systems (ALSA, Pulseaudio, etc.). Also, there were some bugs with loops and repeats. I fixed these bugs, and after testing and talking with my mentor I got that bug with repeats was not my - I haven't broke anything. Anyway, I fixed it.

You can find code of my implementation of JACK Transport reposition here. (I was amending this commit repeatedly, but it still has a date "19th of June". Is it ok?)

If you're interested, you could test it and write me about bugs.

We got a long test of audio drivers from daeavelwyn (Thank you). Looking at tests #2, 5 I could say that it is correct behavior (it's not a joke). We can't use more than one driver at the same time. I've discussed it with my mentor and decided that it is ok: we could use "JACK midi + JACK audio" instead of "JACK midi + ALSA" and "JACK midi + PortAudio". So, now I should change user interface to forbid selecting more than one driver.

When I started to work on this project, I noticed that I have to restart MuseScore if I want to switch an audio driver. It was too annoying and took some time. So, now I can fix it. As my mentor said, "it would be a huge gain of usability".

Also, daeavelwyn gave me another idea about Timebase slave. In short, it will give an ability to change speed of playback with external midi controllers or prerecorded midi track. Here is my retelling of the use case:

Suppose we have a score in MuseScore and an external synth (generates some sounds that couldn't be written as score, for example, Zebra VSTi). We're writing a music for a film, so we're also using a xjadeo. Now suppose we want to send an orchestra hit and a synth sound to underline some action in the film. We want tempo to be a bit slower at this moment because our sync with the movie isn't as good as we would like. We could manage tempo (make a rallentando) in Qtractor - it lets us draw a tempo map, we can slow down or accelerate the music to have a more living/realistic interpretation:
"with this mechanism, there is no limit to the interpretation and if you work on music scores, it's really important to sync everything to the movie" © daeavelwyn

I've already implemented JACK Timebase Master - now JACK Transport and all other application get a signal when tempo changes in MuseScore. To implement described feature I should make an "invert" - read signals about changing tempo in JACK Transport and change tempo in MuseScore. Of course, I am talking now about relative tempo while playback, not about appearing tempo mark in the score.

Key tasks that stalled

Can't solve problem with marked notes. When I seek backward while playbacking, last played chord stays marked (blue notes).

Also I am still getting a crash: open MuseScore and press "Play" on JACK Transport. This worked well before, seems like I broke something.

Upcoming tasks this week

Solve problem with marked notes described above.

Change a UI of "Preferences->I/O": add new checkboxes "JACK Transport", "JACK Timebase Master" and forbid selecting more than one driver.

Implement "Timebase slave" feature described above.

Think about hot switching audio drivers.


As usual, you can find me on IRC #musescore as igevorse.

Also feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·
« Older
Style `onWall HashCode` by Lited & Sayori
Get your own blog immediately for free with Lited!