Shinobu Worker unable to stay on very long #350
Comments
|
LRR Version OS Same issue here, upgraded from 0.7.2 and after a while the Shinobu worker crashes, using the latest nightly but tried with a few 0.7.3 builds and all do the same, downgrading back to 0.7.2 solves the problem. |
|
I've received a similar log through Discord recently -- Are you sure the Shinobu worker is the one thing that's crashing? I haven't implemented a healthcheck for Minion, but you can use the built-in Minion console to check on its status: |
|
Well, it shows 'Kaput' next to Shinobu Status, and the minion console only shows thumbnail errors. I rollbacked my version, which has fixed it for me. |
|
I also thought it was the Minion but I keep the console opened and it showed no errors there. The issue seems to be related to the file watcher since it will stall for a while after this step then crash [Shinobu] [info] Shinobu Background Worker started.
|
|
I might have an idea as to what causes the issue - I'll need to run some tests on the windows version to make sure. |
|
Haven't been able to repro -- I'm pretty sure it's an issue during the filescan shinobu does when started. |
|
|
Thanks for the logs. I've pushed some code to better secure the scan with an eval block for now -- I still don't know the root cause but it might give a clearer error output. |
|
I just tried the latest nightly, still the same error but now it doesn't crash the entire thing at start, here is the log. However, adding any file after that will crash the Shinobu File Watcher
|
|
Hmm, I'm pretty sure now that the error comes from the worker being unable to queue a job in Minion. More |
Difegue
added a commit
that referenced
this issue
Oct 19, 2020
+ retry SQLite open when getting a Minion object
|
Tried the latest nightly, the Shinobu File Watcher doesn't crash anymore after the first scan or when adding new files, thanks. The only thing left are the error messages but I assume that doesn't affect functionality at all? |
|
The errors are still a problem, but it shouldn't affect functionality as-is. The retrying logic I added when trying to get a Minion connection doesn't seem to do anything, sadly, |
|
Just to follow up on this, I decided to add a considerable amount of files at the same time (100+), after the first 40 or so the client stopped adding them to the database but there was no error in the file watcher. I decide to restart the client and during the initial scan it would add 3 or 4 new files to the database, then stop again. This behavior would repeat after every restart. Reverted back to 0.7.2 and all the missing files were added without issue during the first scan. |
|
Damn, this sounds like #282 . The latest nightlies allow to switch to WSL2, could you try with it to see if it improves anything? |
|
Past that, a solution would be to get rid of the SQLite minion backend and switch it to something that works better concurrently -- I wanted to use a Redis backend originally but there wasn't a stable one available in CPAN. Heck, maybe I'll write one myself 🤔 |
|
Tried the latest nightly using WSL2. Added 10 files at the same time, stopped the client after the first one was added and restarted it, same behavior. After the restart it only added 3 new files during the initial scan then stopped, had to restart the client another 2 times until all files were indexed. Let me know if you need logs or something that can help you to reproduce the issue. |
Difegue
added a commit
that referenced
this issue
Oct 26, 2020
|
Oh right I'm dumb, the current I think that by adding more evals there the scan shouldn't stop anymore. |
|
Tried the latest nightly, that did he trick, thanks. |
|
Looks like the metadata plugin timeout is ignored during the opening scan if you restart the client after it stalled, I realized after getting temporally banned in eh. |
Since archive additions work in parallel, you can sometimes get a bunch of metadata plugin hits at the same time despite the timeout. Making a multi-process timer is slightly higher on the complexity side so I didn't do it hahaha 👀 Closing this since the base issue has been fixed with eval duct tape -- See #359 for the real fix, whenever I get to doing it. |
|
what the fuck |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
and others
I restart it, it runs fine for a bit, then it crashes a few seconds later. I'm not sure what to do.
The text was updated successfully, but these errors were encountered: