Hi,
I have a program which stores a lot of text. To save space (it's really a lot of text) I use LZMA compression to reduce the size significantly. Now, compression so much text takes a lot of time and 7-Zip is much faster when using the same option (at least, what I expect to be the same options...).
Without discussing that I could use BriefLZ or split the data or whatever for the sacrifice of some space and other levers I could pull: Can the compression be sped up? I know that the built-in libraries have some more functions that are not mapped into PureBasic directly and from time to time some question is answered by importing this or that function and such. My program is using only one core when compressing, 7-Zip uses all available, so is this maybe already doable?
Accelerate LZMA compression
Accelerate LZMA compression
Good morning, that's a nice tnetennba!
PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD
PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD
Re: Accelerate LZMA compression
I think what you want to use is LZMA2 instead. It's basically the same as LZMA but allows multi-threaded compression. I don't know if this is already included in what comes with PureBasic.
Re: Accelerate LZMA compression
It isn't. Thanks for the explanation, I didn't know that.
Good morning, that's a nice tnetennba!
PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD
PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD
Re: Accelerate LZMA compression
Ok, so I found out that it's not the compression, it's fast enough. The real slowdown is done by ComposeJSON. I want to save a lost containing a structure (which includes basically every data type) with 70000 elements. Is there a better way than jsonizing it? Can I compress the JSON object directly? I'm not so firm with JSON, is it like a memory block, using JSONObjectSize() to get the size?
Good morning, that's a nice tnetennba!
PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD
PureBasic 6.21/Windows 11 x64/Ryzen 7900X/32GB RAM/3TB SSD
Synology DS1821+/DX517, 130.9TB+50.8TB+2TB SSD