The playback note highlight bar really lags…
complete
J
Jesse Rolland
Hey I’m not sure what it’s actually called but the moving highlight bar that highlights each note when played is honestly soooo annoying right now. I feel that it needs to be heavily modified. My issues with it are this. I’m pretty sure it delays by like 1 note. Like I’ve analyzed it carefully and I will notice that it’s not in sync with the metronome at all when I’ll do like triplets. Beyond that the highlight bar has like a mind fucking shadow effect so again this messes me up so much because it really conflicts with the audio metronome. It’s honestly gotten so bad Derrick that I won’t even use the Beat Note playback feature anymore when practicing along. I simply just open up my Sound Brenner metronome app and use Beat Note only to read the music. That’s not a good thing at all. When I am writing triplets with longer rests, watching that highlight cursor bar (what are we calling it?) is chaotic as fuck. I’m a seasoned metronome drummer too. Hopefully you can take a look at it and fine tune it. For now though I can’t practice along with it as by default I’ll continue to use a different metronome app…
Derek Lee
complete
A fix for this issue was included in v3.4.9, released Thurs 6/19/2024. 🎉
I think there are still aspects of the audio/visual integration that could be further improved. The solution I have implemented for this resolves the following two issues I discovered:
#1. Audio loops were causing a minimal loss of time each time the beat looped (we're talking milliseconds). This has been fixed and audio looping is now 100% sample accurate as confirmed through lengthy high-detail tests.
#2. Visualization of the playback (play head) was speeding up approximately 1 frame (16 milliseconds) every minute (when running at a framerate of 60 frames per second). While this was not noticeable with shorter playback times, once playback times reached ~10 minutes (depending on the tempo), it was starting to become noticeable, and usually by 15 minutes or 20 minutes was off by enough that it might be a full 8th note ahead (depending on the tempo).
As with all aspects of the app, I think further improvements are possible and these will be prioritized appropriately with all other new features, enhancements, and bug fixes in the future. 👍🏻
Derek Lee
Quick update for everyone on this - some other users have reported this (outside the roadmap) and raised it as of critical importance to them, and I'm in my third full-week of working on fixing this.
For the developers in the group (I know there are some of you out there), here's the long and short of it:
- There are three components involved in this bug: the sample-accurate audio playback, the play head visualization, and the looping infrastructure.
- After researching this issue, I discovered the primary issue was in the audio playback. This is one of the very few OSS frameworks I use as it helped me get BN development up and running, but unfortunately, it was lacking the functionality I needed for looping. When I implemented looping, I cobbled together an implementation that minimized the changes I'd have to make to this OSS component, and it turns out the symptom of the "lag" is a result of these changes I made.
- I was hoping to resolve the issue entirely in the audio code and I've spent the last two+ weeks working on that. Since it's not my code, and it's highly complex, it's taken an inordinate amount of time for me to come up to speed on it. I'm now at the point where I've been making changes and confirming everything is moving in the right direction. Unfortunately, my hope of fixing this only in the audio layer isn't possible. I also need to completely refactor/re-architect/re-develop the entire playhead visualization and looping mechanisms.
- I continue to make good forward progress on this, but I still am unsure how much longer I have to go. Once this is fixed, however, the audio + playback visualization + looping should be rock-solid. I'm also discovering that through all of this learning, other features that I was unsure about how to implement are becoming a bit more clear.
All other development work is paused while I iron this out. Thanks for your patience. As soon as I have something ready it will go through a short beta test and I will notify of the release. 👍🏻
Derek Lee
Hello friends - I had done some research into this previously and picked this back up as I received another report about it. Here's where I am with this:
- Looping an entire beat on the full beat playback screen (practice screen) appears to be functioning properly. I took a screen recording for 10 minutes and the audio and visual were lined up throughout the entire video.
- When looping a single measure on the practice measure screen (focused practice for a single measure), orwhen using the "loop each measure" function on the practice screen (where each measure is looped X times), it appears that the audio and video do get out of sync. Based on my tests, it looks like the visualization play head is spot-on and accurate based on the selected tempo/bpm, but the audio is falling behind. As a result, you see the playhead hit the note but don't actually hear it until afterwards. For each loop, the audio appears to slow by approximately 90ms or ~5-6 frames. It was about 1 second off after 6+ minutes.
This area of the app is quite complicated and deals with managing events at the level of milliseconds/frames, so it might take me some time to figure out what needs to get tweaked. As soon as I have something I'm confident is an improvement I'll let you know. 👍🏻
Thanks for your patience while I work through this! 🙇🏻♂️
J
Jesse Rolland
Derek Lee yeah I did multiple tests as well (sorry I haven’t sent you a video yet!) and just did another series right now. I am actually getting lag on all three forms: 1) whole beats screen (vertical), 2) single beat (verical), 3) single beat (horizontal). The single beat screens get off within 30 seconds or so easily. The whole beats screen took closer to 3 minutes I’d say. But eventually I am hearing count 1 when the play head is showing the “e” and eventually the “an”. I was having a hard time figuring out whether the audio or the visual is behind since it’s kind of a mind fyck but whatever, it doesn’t really matter. From my experience it gets out of sync.
Derek Lee
Jesse Rolland Thanks for following up on this! Unfortunately, there aren't only "three forms" of playback. There are probably thousands, which is why sending me a video of your experience is the most helpful in debugging issues. I'll list the main ones here in case it's easier for you to list them up;
- What screen? Practice Beat? Or Practice Measure? Both?
- Is it related to the template are you using? Sheet music or single measures worksheet? Or all?
- Repeat settings: Whole Beat or Each Measure? # of times?
- Tempo setting: Constant? Or are you changing the tempo?
- Starting from the beginning of the beat or starting mid-way? Both?
Debugging something like this is needle-in-the-haystack Sherlock Holmes level work. Knowing some of the above parameters helps, but either way - this part of the app is literally millisecond-level adjustments and extremely difficult to find and tune.
I'm testing this with a screen recording to confirm both when the audio is expected to hit and when the highlight occurs (this is at the framerate level, which isn't as accurate as millisecond-level, but it's the best way I've found). At least in the first test I did, everything lined up perfectly (see screenshot below). But there are clearly other configurations that are not working as expected - so any further details you can provide about what settings you're using (or just taking a screen capture works, too!) would be super helpful. 👍🏻
J
Jesse Rolland
Derek Lee ok yeah I will take multiple screen captures of what I am noticing.
Derek Lee
in progress
Derek Lee
planned
Planning to start working on this for an upcoming 3.x release. 👍🏻
Derek Lee
Hi Jesse Rolland and thanks for sharing your feedback. I tend to call it the "visual playhead" but there are probably different names we could use. I'll use that for now.
The visual playhead is a seriously complex part of Beat Note and please believe me when I say I've invested a lot of hours not only in implementing it, but also in ensuring that it is spot-on accurate and in time with the audio. I would not have released it to the public if it wasn't 100% accurate, and based on all of my experience, it has been fine-tuned to be as aligned to the audio as possible and sample accurate. Additionally, this is the first time since the play head was added that anyone has expressed any feedback about it exhibiting the behavior you describe.
Based on your feedback, it sounds like this is not what you are experiencing. In order to troubleshoot this, I'm going to need more information from you and I'm going to need to see what you see. This could be specific to your device, specific to your iOS version, specific to the beat you're using, specific to the tempo you're practicing at - it could be all sorts of things. And please allow me to re-iterate: implementing the visual playhead is an enormous amount of complexity (you really have no idea how complicated it is), so I am really going to need your help to understand what about it is not working properly for you with examples.
For this kind of issue, I'm going to need the following:
- Please send me an email from within the app under Settings > Email Derek. This will contain the technical information about your device.
- Please tell me in exact detail what actions you are taking in the app to consistently recreate this behavior.
- Please tell me the name of the beat or beats where you see this behavior, and the tempo or tempos that show this behavior.
- Please send me a screen recording from your device which illustrates the behavior you are experiencing.
If this information isn't enough, we'll need to set up a time to meet so you can show me. As I hope is already clear to you, the professional quality and robustness of Beat Note is of the utmost importance to me, and I respond as quickly as possible to any issues that are brought to my attention. That said, I am not able to see what you see, so in order to resolve this issue I'm going to need your help.
Thanks for bringing this issue to my attention! I appreciate your help so I can resolve this quickly for you. Your feedback helps make Beat Note better for everyone!
J
Jesse Rolland
Derek Leecool. Yeah Derek I was actually going to first just record a screencast originally for explaining what I am experiencing. In fact I have a screen cast pending right now that I’m going to send to you regarding the “loop brace” feature I’ve requested. But yeah yeah I can imagine the how complexed the play head feature is. I have another app that I am going to use as an example to show you the difference in experiences. And for the record I’m not trying to come across hostile or anything. It’s very clear how much time you’ve invested in this app and my pets ambitions with it are through the roof. I’m just enthusiastic about seeing these constant updates and the continual perfection of Beat Note. There is no other tool like this. But yeah I will go ahead and creat a screen cast and maybe get my videos out to you soon here. Thanks for taking the time to respond to all of our feedback! Love it!
Derek Lee
Jesse Rolland Thanks for your kinds words! I'm pretty sure I have a good idea of how the playhead should be working but I'm happy to take a look at other examples if you think it would be helpful.
I ran some tests today and while the playhead does start perfectly aligned to the audio, and remains there for some time, I noticed two aspects of the behavior that were not performing as accurately as I would have expected (but I also wouldn't consider it "chaotic AF" either lol 😂...)
- Running the playback for a long period of time, the playhead appears to be slowly moving slightly faster than the set tempo. I only noticed this after some # of minutes of continuous practice (I didn't time it explicitly, but it was probably something like over at least 5 minutes). This is strange because the playhead is supposed to be perfectly aligned with the set tempo, so something must be slightly off.
- Adjusting the tempo while the audio is playing appears to not fully process the tempo adjustments accurately and leads to the playhead not being perfectly aligned with the music.
The more examples I have to reference, the more I can understand the behavior you're experiencing and dig into this. I am a perfectionist, so please know that I struggle with finding the right balance between releasing something even if it's not perfect and waiting too long to be too picky and perfectionist about things. Most apps are built by teams, so it's a delicate balancing act for me and I'm trying to find a good balance of pushing stuff out there with the highest level of quality I can muster (you have no idea the complexity that exists under the hood for this lol!) and also getting it out there so you can help me further improve it with your feedback. 🙇🏻♂️ Thanks bud!
Hinrich Nagel
Hi,Derek Lee . Attached you find a link of an example, where the above described behavior (the visual playhead lags behind the sound) occurs after nearly 1 – 2 minutes of playing the first measure on my iPad Pro (12,9“ 6th generation, Apple M2, iPadOS 17.4.1, BN 3.3.2). This is my preferred system which I mostly used. But the same behavior occurs on my MacBookair (15“, Apple M3, MacOS 14.4.1, BN 3.3.2) and, I just tried it out, because I’m interested in, on my iPhone 11 Pro (Apple A13, iOS 17.4.1. BN 3.3.2): the same behavior.
Derek Lee
Hinrich Nagel Thanks for sharing the beat and for the update. I have been able to recreate this behavior on my devices, though there seems to be some variation between different screens and settings. I hope to be able to dig into this and determine a way to improve it. This is one of the most complicated parts of the app, so it will likely take some time for me to find an appropriate solution.
Hinrich Nagel
Derek Lee Good work, congratulations! The app-version 3.4.9. solves the problems which I reported on April, 1th. Thanks a lot.
Derek Lee
Hinrich Nagel Thank you so much for confirming this! I really appreciate you taking the time to let me know. This fix took me a LOT longer to fix than anticipated. I still think there are further improvements to be made, but I want to get back to other important features and updates. If you encounter any other unexpected behavior with this please let me know.
Thanks again for your kindness and support! 🙇🏻♂️ I really appreciate you!