That's all result of subsampling applied. https://en.wikipedia.org/wiki/Chroma_subsampling
Here is an image showing the difference between nicely-done JPEG saving lib and one using default noob way:

The typical and easiest solution is to disable chroma subsampling by setting it's block size to 1x1 pixel, so it does bare calculations and not affects image pixels.
There is big sense to do this, as it surely increases size of JPEG files slightly, but they anyway are taking much less space than PNG or any true lossless format. Thus cool quality/size coefficient is provided to store all your precious photos.
I suggest to add such fix to PB library and some kind of flag to use it optionally when saving files.
PS. There is wrong text on image, used Photoshop JPEG setting was "11 of 12", which is much better than 80 (no matter)