Igevorse Otonnoleare

Realtime midi input: to be or not to be?

08 June 2014 | 11:42 pm

Overview of work this week

In short:

  • Fixed bug with tracking of changing tempo and time signature.
  • Discussed assigning channels to staves.
  • Realtime midi input: possible or not?
  • Exams successfully passed.

There was a bug in my JACK Timebase implementation. Nothing critical. Now it is fixed, and code sent for merging.

I was discussing "assigning channels to staves" feature in mailing lists, and there are some additional things arised:

  1. I was thinking only about assigning channels for JACK MIDI Out, but people need assigning channels for the internal synth too.
  2. It turns out I don't completely understand the picture. I mean internal representation of staves and instruments in code. This week I would dive in, and write a post about these classes and how they're connected.
  3. I thought that I can use "Create Instruments" dialog for assigning channels for the Instruments, but lasconic was right, this window shows Parts, not Instruments. So, do I have to use Mixer dialog? I will answer after diving in (see p.2).

Also, I have an idea of implementing realtime midi input.

In short: user clicks "record" button, plays (with a metronome) something on midi-keyboard, and notes appears on the staves almost immediately.

Pseudo-algorithm:

  1. Wait for NOTE_ON message for notes, save time/tick when got message for every note.
  2. Wait for NOTE_OFF message, save time/tick.
  3. Calc note duration in time/ticks, and calc approximate note duration (half,quarter, etc) according to current tempo.
  4. Look at time of getting NOTE_ON and current time signature. If note does not fit the measure, divide it, add note to the next measure too and make a tie.
  5. If there are several notes pressed in one moment, make a chord. "In one moment" - that's the problem I need to solve. Also, need to find "the end" of chord, making ties for the notes, that continues to sound.

Of course even playing with metronome user wouldn't play notes precisely, so a quarter may be not 480, but 450, 505 or 470 ticks.

Also, there would be a delay because of hardware midi keyboard, I should measure and compensate it.

In general I understand how to make it, but the devil is in the details!

My mentor says that this is almost impossible. Now I don't see details, and think that I can do it. Of course, after finishing adding JACK support in MuseScore. After I came up with this idea, I found that it is already implemented in Finale and Sibelius (I don't use them), but not in the best way. Maybe it is really too hard?

Key tasks that stalled

For implementing this feature I need to connect my midi keyboard (Casio CTK-810) to computer. But I have problems with it. I can use virtual midi keyboard for now, but anyway, it's better to have a physical one.

My system (Ubuntu 13.04) doesn't see it. lsusb doesn't show new device. dmesg shows nothing.aconnect -i and aplay -l doesn't show it too. I updated kernel to 3.8.0-35, but it does no effect. Windows installed on this computer doesn't see the keyboard too.

I checked BIOS settings, tried to change something related to USB, but it doesn't work.

But midi keyboard is working.

I checked on another computer (Ubuntu 13.04 + Windows too), Ubuntu didn't see this keyboard, but I restarted to Windows, it saw my device. I installed drivers and rebooted to Linux. Now Ubuntu see it.

But on my computer not :(.

If you know something about it, write me please.

Upcoming tasks this week

Understand better Staff, Instrument and Parts classes interaction.

Work on assigning channels to staves.

Start working on JACK Transport reposition.

Thank you for giving use cases, examples, and answering questions.

As usual,you can find me on IRC #musescore as igevorse.

Also feel free to ask about implementing some JACK-related features that you need and want.



« JACK Timebase implemented · Realtime midi input: to be or not to be? · JACK Transport: first steps »

  1. Comments (2):


  2. avatar
    Nicolas Froment · 08 June 2014 | 07:36 pm

    Mentor here Wink

    I just want to add that RT midi import is not almost impossible. I don't think I say so. In my opinion, RT MIDI input is worth implementing if it works in 99% of the cases. And implementing it so it works in 50% of the cases would require a lot of effort and time, but wouldn't be worth it. Currently, if users really want to enter MIDI in one pass, they can use a sequencer with a recording mode, possibly edit it and then open the MIDI file in MuseScore. This approach has been improved by last year GSoC student and kill several birds with one stone (MIDI files the user didn't recorded himself can also be opened). The problem being more limited, it still doesn't work in 100% of the case.

    So, as defined in your proposal, I would prefer if you focus on working on Jack during GSoC. Jack MIDI IN in step mode should be implemented and as you mentioned, you still have quite some work to have a well designed port/channel assignment system. Also, it would be great to have Jack Transport working correctly before doing the channel assignment.

  3. avatar
    David Bolton · 10 June 2014 | 05:26 am

    Real-time MIDI input has several difficulties that are not addressed by the straight-forward algorithm you mention. Here's an overview of the difficulties that come to mind right now:

    For single-note instruments (such as wind instruments) the main issues are

    1. Difficulty of playing exactly to the beat with accurate pitches

    2. Difficulty of playing the exact length of notes: repeated notes or large interval jumps are often impossible to play for full value using a MIDI keyboard

    3. Figuring out the correct note spelling for any given note (For complex scores this could be a summer of work unto itself)

    4. The inability to add musical markings as you go. In programs (such as Finale) that let you add almost any musical marking via keyboard shortcut, the inability to add musical markings as you go represents a significant loss of time (since it requires mode switching) and accuracy (since it is easier to miss a marking on a second glance over the notes).

    For multiple-note instruments (such as keyboard), in addition to the difficulties mentioned above:

    1. Difficulty of playing a block chord versus a slight arpeggio, and the difficulty for the computer to interpret a rolled chord versus a series of notes

    2. The habit of pianists shortening note lengths (while holding the pedal) to prepare to play the next chord. This is often a physical requirement to play the next chord on time

    3. The habit of pianists playing a note length longer than written (since it fits the chord and sounds fine)

    4. The difficulty for the computer to interpret whether the written notation should reflect a longer note value because the pedal is down or not.

    5. There are many ways to write out what the pianist played, and frequently the simplest/preferred written form differs slightly from start and release of the notes in practice

    6. Multi-voice staves can be interpreted as a mess of tied notes by real-time input although Andrey Tokarev did a fantastic job of cleaning this up for MIDI import last summer.

    With enough research, coding, and testing perhaps all of these issues can be overcome. However, the real question is real-time MIDI input actually useful.

    Since real-time MIDI input requires accurate pitches and strict adherence to the beat the target audience is well-trained musicians. In all but the simplest cases a well-trained musician can input their music quicker using non-real-time methods (at least in Finale and other scorewriting software that streamline other methods of score entry including full use of keyboard shortcuts for musical markings during note input).

    MuseScore could use some streamlining to speed up the existing score entry methods and bring it up to par with Finale, but it would take considerable resources to streamline real-time note entry to the point that is overtakes other methods of score entry in speed and accuracy.

Leave your response!






Style `onWall HashCode` by Lited & Sayori
Get your own blog immediately for free with Lited!