Igevorse Otonnoleare

Category: Google Summer of Code 2014 · Feed

Seek from JACK Transport: video demo

Overview of work this week

In short:

  • JACK Transport reposition implemented.

If you remember, I was getting a loop between MuseScore and JACK Transport. This was due to several reasons: incomplete understanding of communicating with JACK and slightly incorrect implementation of start/stop from JACK Transport.

This week I was studying sources of applications that use JACK Transport, talking with guys from #jack and correcting my code. Actually, I had to rewrite it from scratch because I went the wrong path last week.

JACK provides 2 methods of sync: for "slow" clients and for the "fast". If application has to read something from disk (imagine 4K video) while seeking, or cannot seek immediately, it is a slow client. Otherwise, it is a fast client. I knew little about working as a fast client, and decided to use a sync callback intended for slow clients. This callback tells us current state an position of JACK Transport.

After talking in #jack (IRC) I understood that using sync callback is wrong in my case - MuseScore is "fast" client because it can seek immediately. In general, both of these approaches work, but we shouldn't use a steam-hammer to crack nuts. Each approach for it's own case.

Being a "fast" client, in a process cycle I should compare JACK Transport's position and position that I expect. If they're not equal, seek to new position. After implementing it I've got a loop problem again, because I wrote new implementation not because of loop problem, but because I should use a right approach.

Implementation of start/stop from JACK Transport was slightly incorrect. It worked well before I began implementing seek feature, but now it makes some difficulties. To be clear: one unnecessary function call can cause several calls. Called functions call several functions again (and the initial too). So, we're getting a loop with increasing amount of function calls every second. I used "lazy" way to fix it - we shouldn't do something if we really don't need to do it. So, I reduced amount of calls for starting, stopping and seeking to JACK Transport and now it works well.

In previous post I described 4 basic cases of using JACK Transport:

  1. Internal seek before playing - clicking on notes and then pressing "play" in MuseScore.
  2. Internal seek while playing - pressing "play" in MuseScore and then clicking on notes.
  3. External seek before playing - seeking via JACK Transport and then pressing "play".
  4. External seek while playing - pressing "play" and seeking via JACK Transport.

You can see the video that shows these cases below (click here to watch on youtube).

I prerecorded video of playing an example score in MuseScore and opened it in qjadeo (GUI front-end to xjadeo). You can see this window on the right of the screen. MuseScore would be in sync with xjadeo.

For seeking from JACK Transport I use Catia and gjacktransport.

Sorry for the little async of audio and video - I had some problems with recording (does anybody know how to record perfectly synced video from x11grab and JACK?).

00:00 - seek from MuseScore while not playing

00:28 - seek from MuseScore while playing

00:41 - seek from Catia while not playing

01:04 - seek from Catia while playing

01:22 - seek from gJackTransport while playing

Key tasks that stalled

None. On the contrary, I moved forward and deepened my knowledge.

Upcoming tasks this week

Basic seeking is implemented now, but there are still some bugs that I need to fix. Seeking does not work correct with repeats now. Seeking with loops works but can be done better. Note after breath sometimes doesn't play.

Also, my implementation couldn't be merged right now because it will break working with other sound-systems (ALSA, etc.). I should fix it and test again.

As usual, you can find me on IRC #musescore as igevorse.

Also feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

JACK Transport: first steps

Overview of work this week

In short:

  • Working on JACK Transport.
  • New ideas about realtime midi input.

JACK Transport! Implementing this is harder than I thought. Actually, it took a week of work.

Implementing JACK Transport seek/reposition consists of several parts:

  1. Internal seek before playing - clicking on notes and then pressing "play" in MuseScore.
  2. Internal seek while playing - pressing "play" in MuseScore and then clicking on notes.
  3. External seek before playing - seeking via JACK Transport and then pressing "play".
  4. External seek while playing - pressing "play" and seeking via JACK Transport.

Now 3/4 of this is done - the first option (internal seek before playing) is not implemented. Everything else works.

I didn't work on implementing "assigning channels for staves" because of work on JACK Transport, but I look through the code and still think about correct implementation of this feature.

Also there were some new ideas about realtime midi input. For example, make variants, different transcriptions of input, from which user can choose the best one. You can see sketch here. This approach helps to make post-editing faster and decrease number of false positives. I want to say thanks to David Bolton, who pointed me to some important difficulties of implementing realtime midi input. There are more and more barriers for making 100% accurate recognition system. So, I underestimated the problem. By the way, realtime midi input is not in my proposal for GSoC, so I don't need to rush with implementing it.

Key tasks that stalled

Internal seek before playing - implementing it breaks another options. Maybe I use the wrong approach? I have tried many approaches this week, current one is the best, I think.

Why does it hard? When we are seeking from MuseScore, we sending a signal to JACK Transport, JACK Transport sends signal to MuseScore to seek, and... we've got a recursion/cycle that runs forever.

Solution is usually very simple, but it is not just a recursion between two functions, we have an interaction between two applications working in several threads. It was not easy to keep all possible interactions in the head. It was new and interesting for me. Of course I will find a solution, just need more time.

Also, I still can't connect my midi keyboard.

Upcoming tasks this week

Finish working on JACK Transport, publish the code.

Test it hard and write about testing.

As usual, you can find me on IRC #musescore as igevorse.

Also feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (1) ·

Realtime midi input: to be or not to be?

Overview of work this week

In short:

  • Fixed bug with tracking of changing tempo and time signature.
  • Discussed assigning channels to staves.
  • Realtime midi input: possible or not?
  • Exams successfully passed.

There was a bug in my JACK Timebase implementation. Nothing critical. Now it is fixed, and code sent for merging.

I was discussing "assigning channels to staves" feature in mailing lists, and there are some additional things arised:

  1. I was thinking only about assigning channels for JACK MIDI Out, but people need assigning channels for the internal synth too.
  2. It turns out I don't completely understand the picture. I mean internal representation of staves and instruments in code. This week I would dive in, and write a post about these classes and how they're connected.
  3. I thought that I can use "Create Instruments" dialog for assigning channels for the Instruments, but lasconic was right, this window shows Parts, not Instruments. So, do I have to use Mixer dialog? I will answer after diving in (see p.2).

Also, I have an idea of implementing realtime midi input.

In short: user clicks "record" button, plays (with a metronome) something on midi-keyboard, and notes appears on the staves almost immediately.


  1. Wait for NOTE_ON message for notes, save time/tick when got message for every note.
  2. Wait for NOTE_OFF message, save time/tick.
  3. Calc note duration in time/ticks, and calc approximate note duration (half,quarter, etc) according to current tempo.
  4. Look at time of getting NOTE_ON and current time signature. If note does not fit the measure, divide it, add note to the next measure too and make a tie.
  5. If there are several notes pressed in one moment, make a chord. "In one moment" - that's the problem I need to solve. Also, need to find "the end" of chord, making ties for the notes, that continues to sound.

Of course even playing with metronome user wouldn't play notes precisely, so a quarter may be not 480, but 450, 505 or 470 ticks.

Also, there would be a delay because of hardware midi keyboard, I should measure and compensate it.

In general I understand how to make it, but the devil is in the details!

My mentor says that this is almost impossible. Now I don't see details, and think that I can do it. Of course, after finishing adding JACK support in MuseScore. After I came up with this idea, I found that it is already implemented in Finale and Sibelius (I don't use them), but not in the best way. Maybe it is really too hard?

Key tasks that stalled

For implementing this feature I need to connect my midi keyboard (Casio CTK-810) to computer. But I have problems with it. I can use virtual midi keyboard for now, but anyway, it's better to have a physical one.

My system (Ubuntu 13.04) doesn't see it. lsusb doesn't show new device. dmesg shows nothing.aconnect -i and aplay -l doesn't show it too. I updated kernel to 3.8.0-35, but it does no effect. Windows installed on this computer doesn't see the keyboard too.

I checked BIOS settings, tried to change something related to USB, but it doesn't work.

But midi keyboard is working.

I checked on another computer (Ubuntu 13.04 + Windows too), Ubuntu didn't see this keyboard, but I restarted to Windows, it saw my device. I installed drivers and rebooted to Linux. Now Ubuntu see it.

But on my computer not :(.

If you know something about it, write me please.

Upcoming tasks this week

Understand better Staff, Instrument and Parts classes interaction.

Work on assigning channels to staves.

Start working on JACK Transport reposition.

Thank you for giving use cases, examples, and answering questions.

As usual,you can find me on IRC #musescore as igevorse.

Also feel free to ask about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (2) ·

JACK Timebase implemented

Overview of work this week

In short:

  • Fixed bugs and crashes.
  • Published implementation of JACK Timebase.
  • Began working on selecting channels for staves for JACK MIDI Out.

I found a few bugs and fixed it - I'll talk about this later.

I began working on JACK Transport, as I said last week.

Features that we need from JACK Transport:

1. Play/stop

2. Reposition (locate/seek)

3. Timebase

The Play/stop feature was already implemented, but most of fixed bugs are related to this feature. Repositioning never was implemented, so it need to be done from scratch.

Timebase was implemented this week. With timebase callback we can tell our current tempo, time signature and beat position to JACK.

Also I began working on "selecting channels for staves" feature. This feature is necessary for better routing midi-signals, controlling VST(i) and making MuseScore more professional. People asking for this feature for years (from 2011 I think), so I hope they would be happy Smile.

I've got use cases from users, wrote to MuseScore developer mailing lists about it and outlined possible solutions:

1. Add spinbox to mixer to change channels of each stave (show it only if Preferences->I/O->"Use JACK MIDI" checkbox is checked).

2. Create a port for each staff.

Most likely there is N port limit in different OS (Windows, Linux, Mac), so the second solution is failing.

The first one is not the best too, but could be improved. For example, change channels not in Mixer, but in the Instrument dialogue. So, there are options I need to consider.

If you have some proposals or advices about this feature - you're welcome to discussion.

Also feel free to ask about implementing some JACK-related features that you need and want.

You can find me on IRC #musescore as igevorse.

Key tasks that stalled

I had a beautiful time of finding a causes of crashes. The solution is fairly simple, but to find it I had lots of debug hours.

But now it's OK.

Upcoming tasks this week

Discuss assigning channels to staves.

Work on JACK Transport reposition (locate).

Also I want to mention that the next week would be the last week of my exams, and the hardest one. So, maybe there wouldn't be any code at all :( .

· Retweet Retweet! · Share Share · Comments (1) ·

JACK MIDI OUT: now published

Overview of work this week

In short:

  • I've fixed old and new bugs.
  • Published basic implementation of JACK MIDI Out.
  • I've began to study time representation in MuseScore and JACK.

So, I've fixed bugs mentioned in previous post, found new bugs and fixed them too. Code of MuseScore 1.3 that works with JACK have some bugs and it is cool that my work is not only writing new code, but fixing an old bugs too. Why? Because it helps to understand the code better.

I've learned how time is represented in MuseScore. It's really simple: everything is ticks. All things are counted in ticks. With the help of ticks we can represent beats. Each beat is 480 ticks. So, a 4/4 measure will have a 4*480 ticks and so on. Understanding of time in MuseScore is important for building a JACK Transport support.

In previous post I mentioned "learning about channels and instruments" as "upcoming task", but I decided to postpone it. I think now I should make a basic support of JACK MIDI OUT and JACK Transport, and only then improve it. JACK Transport now could start and stop, but not seek. Also, I should implement an ability to exchange some meta information with JACK, such as tempo, time signature, etc.

Key tasks that stalled

There is no things that scare me, I have some problems or questions periodically, and I solve it as I can. If I can't solve it, I'm talking to my mentor and solving it anyway Smile.

Upcoming tasks this week

Study more about time in MuseScore and JACK, begin working on JACK Transport.

· Retweet Retweet! · Share Share · Comments (0) ·

Basic MIDI output: done!

Overview of work this week

In short:

  • Started diving in MuseScore 1.2 and 2.0 code.
  • Now we have basic output of midi signals via JACK MIDI.
  • Found a few bugs.

Unfortunately I have an exams in my university now, so I can't write lots of code. But it does not prevent me from research and learn the existing code.

Previously I used JACK only as user, don't trying to understand how it works exactly. But this week I had to dive in as a programmer. Also I had to learn more information about midi and midi messages.

I looked through the MuseScore 1.2 code and saw how JACK MIDI code was working before and compared with 2.0.

A great help for me is that there is some code saved from previous versions of MuseScore. I wrote a bit of code that helps to run this existing code (slightly changed) and now we have a basic outputting if midi signals via JACK MIDI. Really, it looks like a buggy implementation of 1.2, but it's a good starting point.

I said "buggy" - I found a few bugs (one of them was in 1.2 already ) and now need to fix them Kickboxing. Now only I work on this part of MuseScore, so I think there is no reason to write to the issue tracker. You could see an information about bugs in unpublished code here.

Key tasks that stalled

I have some misunderstanding about channels and instruments interactions, but I have a good starting point for learning.

Upcoming tasks this week

I know there are people waiting for JACK MIDI OUT, so I'll try publish existing code. But before I need to fix bugs mentioned above.

Also I will clean an existing code saved from MuseScore 1.2 - some parts now can be useless.

Also, as I said, I will learn more about channels and instruments.

· Retweet Retweet! · Share Share · Comments (0) ·



I am a participant of Google Summer of Code 2014, and in this blog I will describe progress of my project.

My goal is to implement a full JACK support in MuseScore.

JACK Audio Connection Kit is a professional sound server daemon that provides real-time, low latency connections for both audio and MIDI data between applications that implement its API.

To be specific, I will be adding the currently missing JACK MIDI I/O and JACK Transport support to the application.

This will be a great feature that gives an ability to connect with lots of synthesizers and Digital Audio Workstations. In general, it will be possible to route MIDI and Audio signals, to do a real-time sound effects processing and a lot of other cool things.

You can see the progress on my github page.

Current task: familiarize with the existing code.

· Retweet Retweet! · Share Share · Comments (0) ·
Newer »
Style `onWall HashCode` by Lited & Sayori
Get your own blog immediately for free with Lited!