New Generator Testing (UPDATE)

1 post / 0 new
puravida's picture
Jedi Warrior
Joined: 09/01/2007
Visit puravida's Website


This is an update on our progress regarding the new generator testing. It has been several months since we began to see an increase in failed captures (up to 0.0017% failure rate). Over the past month, we have been monitoring a minor update we released that began noting certain types of failures as FILE_CREATE_ERROR. After much research and debugging, the only common ground we could identify was that most cases of FILE_CREATE_ERROR involved websites that had very large images that were scaled down (not best practice in web design but we should still support it) or used as a background. That really shouldn't matter but, after much debugging and investigation, we cannot determine any reason for the failures. The system simply crashes with very little resource usage; plenty of free RAM and CPU.

Since there are a million working parts, a buggy variant could have come into play anywhere in the process and we cannot determine where. Therefore, we decided to revert back to a known working version of our capture generator codebase and then work forward from there. In early testing, we have resolved 99.999% of the FILE_CREATE_ERROR conditions and the newer version even resolves a portion of the CAPTURE_FAILED conditions.

Overall, the new generator version should be more stable and reduce error percentage from a recent 0.0017% failure rate down to 0.00001%. These might sound like low percentages already and many may not have noticed the uptick in errors, but it makes a difference when working with a large database of URLs or when the failure happens for a well-known website.

We are working diligently to continue to improve our service and keep it running optimally. Over the next week or two, we should begin rolling out the updated generators. There should be no impact or downtime and no action is required on your part.

ShrinkTheWeb® (About STW) is another innovation by Neosys Consulting
Contact Us | PagePix Benefits | Learn More | STW Forums | Our Partners | Privacy Policy | Terms of Use

Announcing Javvy, the best crypto exchange and wallet solution (coming soon!)

©2022 ShrinkTheWeb. All rights reserved. ShrinkTheWeb is a registered trademark of ShrinkTheWeb.