Igevorse Otonnoleare

Bugs & magic with cross-thread function calls

Overview of work this week

In short:

  • I was fixing bugs.

Yes, it is very simple. I just fixed bugs.

I found these bugs (8-12) while working on JACK Transport.

I did not work with count in or tempo slider in play panel, but I've implemented JACK reposition, and these features used my new code.

But I spent a lot of time on the second-to-last bug (11 on the "Bugs" page). It seems like very simple: I was getting a crash when I press "Play" on JACK Transport and there is time 0:00. I've checked all my code that implements reposition from JACK Transport and found the cause: uninitialized iterator Seq::guiPos. MuseScore crashes when was touching it in HeartBeatTimeout() - this function updates the cursor on the screen.

This iterator is initialized explicitly in seek() and we calling seek() when start playing. So, what's the problem? The problem was - I think you already got it - in my implementation. Seq::start() does not always call seek() because of some conditions I wrote. The solution is quite simple: I should change or roll back conditions. But in this case I will get JACK Transport reposition non-working.

Finally, I got it to work, but caught an another bug, very strange one. That's where the magic begins. In some cases MuseScore crashed with different errors about memory management, sometimes canvas didn't repaint. Actually, I spent most of the time on fixing these two bugs.

When I realized that I was stuck I wrote to my mentor and he gave me an idea: I did not use seek() function correctly. Really, I didn't think about threads a lot and used seek() wherever I wanted. Seek() calls functions that should be called only from GUI thread, but I called seek() from realtime thread.

So, the solution is splitting seek() into two functions: the first should be called from the GUI thread, and the second - from the realtime thread. Now we have seek() and seekRT().

After fixing the last bug I've checked all new features again and made a pull request.

Key tasks that stalled

Cross-thread function calls. I wrote above.

Upcoming tasks this week

Implement "Assigning channels to staves" feature I described in first posts.

Hot switching drivers is partially working, so there is no need to hurry up with it.


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

New UI: improved usability

Overview of work this week

In short:

  • Implemented JACK Timebase slave.
  • Implemented hot changing parameters of JACK.
  • Changed Preferences->I/O UI.
  • Implemented remembering/restoring JACK audio connections.
  • Fixed bug with repeats. Found new bugs.

I've implemented feature requested by daeavelwyn: JACK Timebase slave. In short, this feature allows MuseScore to sync to JACK's tempo. Now you can change tempo in application (current Timebase Master) connected to JACK and MuseScore will change internal tempo too. Of course, I am talking about relative tempo now, it applies only to playback tempo and doesn't change tempo marks in the score.

Anytime you can explicitly make MuseScore a Timebase Master and control tempo from MuseScore. This is possible now because of the second implemented feature: changing parameters of JACK driver without unloading it and restarting program. Now you can open Preferences->I/O, change preferences in "JACK audio server" section, click "Apply" or "Ok" and continue work on your score. In general, it gives you an ability to:

  1. Enable/disable JACK Audio and JACK Midi.
  2. Change number of midi ports dynamically (increase, decrease).
  3. Become a Timebase Master or not (new checkbox "Timebase Master").
  4. Enable JACK Transport or be independent (new checkbox "Use JACK Transport").
  5. Turn on/off remembering connections.

As you understand, I had to slightly change the Preferences->I/O window. Now user can not select more than one driver.

Also I've implemented remembering/restoring JACK audio connections. It was already partially implemented, MuseScore remembered connections not from patchbay, but connections made in Preferences->I/O. I removed extra controls and now MuseScore remembers connections you made in patchbay. There is no limit to number of ports. Also, if you disable "Use JACK Audio", but stay enabled "Remember connection(s)", the next time you enable JACK Audio, ports would be auto-connected. The same thing with MIDI ports.

I fixed bug with repeats mentioned in previous post, but found new. The first bug related to "count in" feature: JACK transport begin playing in the same time with counts, but should begin after count in ends.

The second bug related to Play panel and tempo slider. When changing tempo with this slider, MuseScore keeps current position in uticks, but not in samples (number of current sample changes, because it related to the tempo). So, I got seeking forward when increasing tempo, and seek backward when decreasing it.

But never fear - these errors not in master branch, only in my code. However, as a new features too... All described features are in jack-transport branch now, I'll make a pull request only after fixing all bugs.

Key tasks that stalled

All is well.

Upcoming tasks this week

Fix new bugs.

Fix old bugs (marked notes, crash on start - mentioned in previous post).

Think more about hot switching audio drivers.


As usual, you can find me on IRC #musescore as igevorse.

Feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

JACK Transport: code published

Overview of work this week

In short:

  • JACK Transport reposition: code published.
  • Found two bugs with repeats.
  • New ideas about Timebase slave.
  • Ideas about hot switching of audio drivers.

As i wrote in previous post, my code of JACK Transport reposition worked, but wasn't ready for merging and could break working with other sound-systems (ALSA, Pulseaudio, etc.). Also, there were some bugs with loops and repeats. I fixed these bugs, and after testing and talking with my mentor I got that bug with repeats was not my - I haven't broke anything. Anyway, I fixed it.

You can find code of my implementation of JACK Transport reposition here. (I was amending this commit repeatedly, but it still has a date "19th of June". Is it ok?)

If you're interested, you could test it and write me about bugs.

We got a long test of audio drivers from daeavelwyn (Thank you). Looking at tests #2, 5 I could say that it is correct behavior (it's not a joke). We can't use more than one driver at the same time. I've discussed it with my mentor and decided that it is ok: we could use "JACK midi + JACK audio" instead of "JACK midi + ALSA" and "JACK midi + PortAudio". So, now I should change user interface to forbid selecting more than one driver.

When I started to work on this project, I noticed that I have to restart MuseScore if I want to switch an audio driver. It was too annoying and took some time. So, now I can fix it. As my mentor said, "it would be a huge gain of usability".

Also, daeavelwyn gave me another idea about Timebase slave. In short, it will give an ability to change speed of playback with external midi controllers or prerecorded midi track. Here is my retelling of the use case:

Suppose we have a score in MuseScore and an external synth (generates some sounds that couldn't be written as score, for example, Zebra VSTi). We're writing a music for a film, so we're also using a xjadeo. Now suppose we want to send an orchestra hit and a synth sound to underline some action in the film. We want tempo to be a bit slower at this moment because our sync with the movie isn't as good as we would like. We could manage tempo (make a rallentando) in Qtractor - it lets us draw a tempo map, we can slow down or accelerate the music to have a more living/realistic interpretation:
"with this mechanism, there is no limit to the interpretation and if you work on music scores, it's really important to sync everything to the movie" © daeavelwyn

I've already implemented JACK Timebase Master - now JACK Transport and all other application get a signal when tempo changes in MuseScore. To implement described feature I should make an "invert" - read signals about changing tempo in JACK Transport and change tempo in MuseScore. Of course, I am talking now about relative tempo while playback, not about appearing tempo mark in the score.

Key tasks that stalled

Can't solve problem with marked notes. When I seek backward while playbacking, last played chord stays marked (blue notes).

Also I am still getting a crash: open MuseScore and press "Play" on JACK Transport. This worked well before, seems like I broke something.

Upcoming tasks this week

Solve problem with marked notes described above.

Change a UI of "Preferences->I/O": add new checkboxes "JACK Transport", "JACK Timebase Master" and forbid selecting more than one driver.

Implement "Timebase slave" feature described above.

Think about hot switching audio drivers.


As usual, you can find me on IRC #musescore as igevorse.

Also feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

Seek from JACK Transport: video demo

Overview of work this week

In short:

  • JACK Transport reposition implemented.

If you remember, I was getting a loop between MuseScore and JACK Transport. This was due to several reasons: incomplete understanding of communicating with JACK and slightly incorrect implementation of start/stop from JACK Transport.

This week I was studying sources of applications that use JACK Transport, talking with guys from #jack and correcting my code. Actually, I had to rewrite it from scratch because I went the wrong path last week.

JACK provides 2 methods of sync: for "slow" clients and for the "fast". If application has to read something from disk (imagine 4K video) while seeking, or cannot seek immediately, it is a slow client. Otherwise, it is a fast client. I knew little about working as a fast client, and decided to use a sync callback intended for slow clients. This callback tells us current state an position of JACK Transport.

After talking in #jack (IRC) I understood that using sync callback is wrong in my case - MuseScore is "fast" client because it can seek immediately. In general, both of these approaches work, but we shouldn't use a steam-hammer to crack nuts. Each approach for it's own case.

Being a "fast" client, in a process cycle I should compare JACK Transport's position and position that I expect. If they're not equal, seek to new position. After implementing it I've got a loop problem again, because I wrote new implementation not because of loop problem, but because I should use a right approach.

Implementation of start/stop from JACK Transport was slightly incorrect. It worked well before I began implementing seek feature, but now it makes some difficulties. To be clear: one unnecessary function call can cause several calls. Called functions call several functions again (and the initial too). So, we're getting a loop with increasing amount of function calls every second. I used "lazy" way to fix it - we shouldn't do something if we really don't need to do it. So, I reduced amount of calls for starting, stopping and seeking to JACK Transport and now it works well.

In previous post I described 4 basic cases of using JACK Transport:

  1. Internal seek before playing - clicking on notes and then pressing "play" in MuseScore.
  2. Internal seek while playing - pressing "play" in MuseScore and then clicking on notes.
  3. External seek before playing - seeking via JACK Transport and then pressing "play".
  4. External seek while playing - pressing "play" and seeking via JACK Transport.

You can see the video that shows these cases below (click here to watch on youtube).

I prerecorded video of playing an example score in MuseScore and opened it in qjadeo (GUI front-end to xjadeo). You can see this window on the right of the screen. MuseScore would be in sync with xjadeo.

For seeking from JACK Transport I use Catia and gjacktransport.

Sorry for the little async of audio and video - I had some problems with recording (does anybody know how to record perfectly synced video from x11grab and JACK?).

00:00 - seek from MuseScore while not playing

00:28 - seek from MuseScore while playing

00:41 - seek from Catia while not playing

01:04 - seek from Catia while playing

01:22 - seek from gJackTransport while playing

Key tasks that stalled

None. On the contrary, I moved forward and deepened my knowledge.

Upcoming tasks this week

Basic seeking is implemented now, but there are still some bugs that I need to fix. Seeking does not work correct with repeats now. Seeking with loops works but can be done better. Note after breath sometimes doesn't play.

Also, my implementation couldn't be merged right now because it will break working with other sound-systems (ALSA, etc.). I should fix it and test again.

As usual, you can find me on IRC #musescore as igevorse.

Also feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (0) ·

JACK Transport: first steps

Overview of work this week

In short:

  • Working on JACK Transport.
  • New ideas about realtime midi input.

JACK Transport! Implementing this is harder than I thought. Actually, it took a week of work.

Implementing JACK Transport seek/reposition consists of several parts:

  1. Internal seek before playing - clicking on notes and then pressing "play" in MuseScore.
  2. Internal seek while playing - pressing "play" in MuseScore and then clicking on notes.
  3. External seek before playing - seeking via JACK Transport and then pressing "play".
  4. External seek while playing - pressing "play" and seeking via JACK Transport.

Now 3/4 of this is done - the first option (internal seek before playing) is not implemented. Everything else works.

I didn't work on implementing "assigning channels for staves" because of work on JACK Transport, but I look through the code and still think about correct implementation of this feature.

Also there were some new ideas about realtime midi input. For example, make variants, different transcriptions of input, from which user can choose the best one. You can see sketch here. This approach helps to make post-editing faster and decrease number of false positives. I want to say thanks to David Bolton, who pointed me to some important difficulties of implementing realtime midi input. There are more and more barriers for making 100% accurate recognition system. So, I underestimated the problem. By the way, realtime midi input is not in my proposal for GSoC, so I don't need to rush with implementing it.

Key tasks that stalled

Internal seek before playing - implementing it breaks another options. Maybe I use the wrong approach? I have tried many approaches this week, current one is the best, I think.

Why does it hard? When we are seeking from MuseScore, we sending a signal to JACK Transport, JACK Transport sends signal to MuseScore to seek, and... we've got a recursion/cycle that runs forever.

Solution is usually very simple, but it is not just a recursion between two functions, we have an interaction between two applications working in several threads. It was not easy to keep all possible interactions in the head. It was new and interesting for me. Of course I will find a solution, just need more time.

Also, I still can't connect my midi keyboard.

Upcoming tasks this week

Finish working on JACK Transport, publish the code.

Test it hard and write about testing.

As usual, you can find me on IRC #musescore as igevorse.

Also feel free to write me about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (1) ·

Realtime midi input: to be or not to be?

Overview of work this week

In short:

  • Fixed bug with tracking of changing tempo and time signature.
  • Discussed assigning channels to staves.
  • Realtime midi input: possible or not?
  • Exams successfully passed.

There was a bug in my JACK Timebase implementation. Nothing critical. Now it is fixed, and code sent for merging.

I was discussing "assigning channels to staves" feature in mailing lists, and there are some additional things arised:

  1. I was thinking only about assigning channels for JACK MIDI Out, but people need assigning channels for the internal synth too.
  2. It turns out I don't completely understand the picture. I mean internal representation of staves and instruments in code. This week I would dive in, and write a post about these classes and how they're connected.
  3. I thought that I can use "Create Instruments" dialog for assigning channels for the Instruments, but lasconic was right, this window shows Parts, not Instruments. So, do I have to use Mixer dialog? I will answer after diving in (see p.2).

Also, I have an idea of implementing realtime midi input.

In short: user clicks "record" button, plays (with a metronome) something on midi-keyboard, and notes appears on the staves almost immediately.

Pseudo-algorithm:

  1. Wait for NOTE_ON message for notes, save time/tick when got message for every note.
  2. Wait for NOTE_OFF message, save time/tick.
  3. Calc note duration in time/ticks, and calc approximate note duration (half,quarter, etc) according to current tempo.
  4. Look at time of getting NOTE_ON and current time signature. If note does not fit the measure, divide it, add note to the next measure too and make a tie.
  5. If there are several notes pressed in one moment, make a chord. "In one moment" - that's the problem I need to solve. Also, need to find "the end" of chord, making ties for the notes, that continues to sound.

Of course even playing with metronome user wouldn't play notes precisely, so a quarter may be not 480, but 450, 505 or 470 ticks.

Also, there would be a delay because of hardware midi keyboard, I should measure and compensate it.

In general I understand how to make it, but the devil is in the details!

My mentor says that this is almost impossible. Now I don't see details, and think that I can do it. Of course, after finishing adding JACK support in MuseScore. After I came up with this idea, I found that it is already implemented in Finale and Sibelius (I don't use them), but not in the best way. Maybe it is really too hard?

Key tasks that stalled

For implementing this feature I need to connect my midi keyboard (Casio CTK-810) to computer. But I have problems with it. I can use virtual midi keyboard for now, but anyway, it's better to have a physical one.

My system (Ubuntu 13.04) doesn't see it. lsusb doesn't show new device. dmesg shows nothing.aconnect -i and aplay -l doesn't show it too. I updated kernel to 3.8.0-35, but it does no effect. Windows installed on this computer doesn't see the keyboard too.

I checked BIOS settings, tried to change something related to USB, but it doesn't work.

But midi keyboard is working.

I checked on another computer (Ubuntu 13.04 + Windows too), Ubuntu didn't see this keyboard, but I restarted to Windows, it saw my device. I installed drivers and rebooted to Linux. Now Ubuntu see it.

But on my computer not :(.

If you know something about it, write me please.

Upcoming tasks this week

Understand better Staff, Instrument and Parts classes interaction.

Work on assigning channels to staves.

Start working on JACK Transport reposition.

Thank you for giving use cases, examples, and answering questions.

As usual,you can find me on IRC #musescore as igevorse.

Also feel free to ask about implementing some JACK-related features that you need and want.

· Retweet Retweet! · Share Share · Comments (2) ·

JACK Timebase implemented

Overview of work this week

In short:

  • Fixed bugs and crashes.
  • Published implementation of JACK Timebase.
  • Began working on selecting channels for staves for JACK MIDI Out.

I found a few bugs and fixed it - I'll talk about this later.

I began working on JACK Transport, as I said last week.

Features that we need from JACK Transport:

1. Play/stop

2. Reposition (locate/seek)

3. Timebase

The Play/stop feature was already implemented, but most of fixed bugs are related to this feature. Repositioning never was implemented, so it need to be done from scratch.

Timebase was implemented this week. With timebase callback we can tell our current tempo, time signature and beat position to JACK.

Also I began working on "selecting channels for staves" feature. This feature is necessary for better routing midi-signals, controlling VST(i) and making MuseScore more professional. People asking for this feature for years (from 2011 I think), so I hope they would be happy Smile.

I've got use cases from users, wrote to MuseScore developer mailing lists about it and outlined possible solutions:

1. Add spinbox to mixer to change channels of each stave (show it only if Preferences->I/O->"Use JACK MIDI" checkbox is checked).

2. Create a port for each staff.

Most likely there is N port limit in different OS (Windows, Linux, Mac), so the second solution is failing.

The first one is not the best too, but could be improved. For example, change channels not in Mixer, but in the Instrument dialogue. So, there are options I need to consider.

If you have some proposals or advices about this feature - you're welcome to discussion.

Also feel free to ask about implementing some JACK-related features that you need and want.

You can find me on IRC #musescore as igevorse.

Key tasks that stalled

I had a beautiful time of finding a causes of crashes. The solution is fairly simple, but to find it I had lots of debug hours.

But now it's OK.

Upcoming tasks this week

Discuss assigning channels to staves.

Work on JACK Transport reposition (locate).

Also I want to mention that the next week would be the last week of my exams, and the hardest one. So, maybe there wouldn't be any code at all :( .

· Retweet Retweet! · Share Share · Comments (1) ·

JACK MIDI OUT: now published

Overview of work this week

In short:

  • I've fixed old and new bugs.
  • Published basic implementation of JACK MIDI Out.
  • I've began to study time representation in MuseScore and JACK.

So, I've fixed bugs mentioned in previous post, found new bugs and fixed them too. Code of MuseScore 1.3 that works with JACK have some bugs and it is cool that my work is not only writing new code, but fixing an old bugs too. Why? Because it helps to understand the code better.

I've learned how time is represented in MuseScore. It's really simple: everything is ticks. All things are counted in ticks. With the help of ticks we can represent beats. Each beat is 480 ticks. So, a 4/4 measure will have a 4*480 ticks and so on. Understanding of time in MuseScore is important for building a JACK Transport support.

In previous post I mentioned "learning about channels and instruments" as "upcoming task", but I decided to postpone it. I think now I should make a basic support of JACK MIDI OUT and JACK Transport, and only then improve it. JACK Transport now could start and stop, but not seek. Also, I should implement an ability to exchange some meta information with JACK, such as tempo, time signature, etc.

Key tasks that stalled

There is no things that scare me, I have some problems or questions periodically, and I solve it as I can. If I can't solve it, I'm talking to my mentor and solving it anyway Smile.

Upcoming tasks this week

Study more about time in MuseScore and JACK, begin working on JACK Transport.

· Retweet Retweet! · Share Share · Comments (0) ·

Basic MIDI output: done!

Overview of work this week

In short:

  • Started diving in MuseScore 1.2 and 2.0 code.
  • Now we have basic output of midi signals via JACK MIDI.
  • Found a few bugs.

Unfortunately I have an exams in my university now, so I can't write lots of code. But it does not prevent me from research and learn the existing code.

Previously I used JACK only as user, don't trying to understand how it works exactly. But this week I had to dive in as a programmer. Also I had to learn more information about midi and midi messages.

I looked through the MuseScore 1.2 code and saw how JACK MIDI code was working before and compared with 2.0.

A great help for me is that there is some code saved from previous versions of MuseScore. I wrote a bit of code that helps to run this existing code (slightly changed) and now we have a basic outputting if midi signals via JACK MIDI. Really, it looks like a buggy implementation of 1.2, but it's a good starting point.

I said "buggy" - I found a few bugs (one of them was in 1.2 already ) and now need to fix them Kickboxing. Now only I work on this part of MuseScore, so I think there is no reason to write to the issue tracker. You could see an information about bugs in unpublished code here.

Key tasks that stalled

I have some misunderstanding about channels and instruments interactions, but I have a good starting point for learning.

Upcoming tasks this week

I know there are people waiting for JACK MIDI OUT, so I'll try publish existing code. But before I need to fix bugs mentioned above.

Also I will clean an existing code saved from MuseScore 1.2 - some parts now can be useless.

Also, as I said, I will learn more about channels and instruments.

· Retweet Retweet! · Share Share · Comments (0) ·

Welcome

Hi!

I am a participant of Google Summer of Code 2014, and in this blog I will describe progress of my project.

My goal is to implement a full JACK support in MuseScore.

JACK Audio Connection Kit is a professional sound server daemon that provides real-time, low latency connections for both audio and MIDI data between applications that implement its API.

To be specific, I will be adding the currently missing JACK MIDI I/O and JACK Transport support to the application.

This will be a great feature that gives an ability to connect with lots of synthesizers and Digital Audio Workstations. In general, it will be possible to route MIDI and Audio signals, to do a real-time sound effects processing and a lot of other cool things.

You can see the progress on my github page.

Current task: familiarize with the existing code.

· Retweet Retweet! · Share Share · Comments (0) ·
Newer »
Style `onWall HashCode` by Lited & Sayori
Get your own blog immediately for free with Lited!