r/pythonhelp • u/Ok_Aioli2605 • 11d ago
Mr.Veeraragavan
I am loading few large size binary files (70 MB19 nos. look_up tables) in Main using more than 2/3 python files. On completion of the loading, I have to take up a series of search (in 3 or more threads) and arrive at the solution. I am using array.fromfile for loading which does not wait for loading completion. Hence, it fails saying array out of bounds. How do I delay the start of search, just till the loading is completed.
2
1
u/dragonvibes01 5d ago
array.fromfile() is synchronous in the thread that calls it; it does not launch a background load itself. If your loading functions run in separate threads, wait for those threads to finish before starting any searches (join each loader thread, or call result() on every loader Future so loading errors also propagate). An arbitrary sleep would not reliably fix that. Also check the expected item count, array typecode/itemsize and the failing index: fromfile(f, n) reads n items, not n bytes, and EOFError can leave a partially filled array. The docs describe that behavior here: https://docs.python.org/3/library/array.html#array.array.fromfile . Could you share a minimal version of how you start the loaders/search threads and the full traceback, using dummy data? Without that code, a loading race is only one possibility, not a confirmed cause.
•
u/AutoModerator 11d ago
To give us the best chance to help you, please include any relevant code.
Note. Please do not submit images of your code. Instead, for shorter code you can use Reddit markdown (4 spaces or backticks, see this Formatting Guide). If you have formatting issues or want to post longer sections of code, please use Privatebin, GitHub or Compiler Explorer.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.