Plone need to reach a new QA level: Adding nightly performance tests. It is important when you make a modification on the code to know if there is a impact on the performance of the application. As Elisabeth said in the “unloading plone” conference, measure !
To understand and use technologies, you first need to know what are the names of all these packages:
- Funkload Functional and load web tester
- Buildbot System to automate the compile/test cycle
- collective.coreloadtests Three funkload tests for Plone + some setup profiles
- collective.loadtesting One buildout to setup a buildbot with funkload + some examples.
- collective.funkload Complex functional load testing and benchmarking
- collective.buildbot A set of zc.buildout recipes and support for declarative configuration for Buildbot
- collective.recipe.funkload Easy running of funkload tests and generation of reports.
So what has been done during the sprint:
Day One
I have reviewed pasteFunBot. I have kept some scripts and the template script but I have removed all the templates.
I have read the code of most of the differents eggs and tryed most of them.
What I had in mind before starting:
- Buildbot + Plone4 + Funkload + coreloadtests
- Use collective.loadtesting as an example
- Add a paster script to start continuous performance tests faster
By the end of the day 1 I have done collective.funkbot a paster that create a new performance test bot in three questions:
- vcs [‘svn’]
- vcs_url [”]
- email [”]
In the Plone community we are using buildout, so the vcs_url is supposed to be the buildout url to checkout / update every night. email is the email adress where will be sent the reports. Also if the buildbot failed to build the buildout, a mail with all stdio will be sent.
At the end of the day I had some troubles with funkload. I was not able to launch performance tests.
Day Two
In the morning tomlazar and vincinc has helped me and I have disovered their work on collective.recipe.funkload done during the last performance sprint in Bristol. In fact funkload script need to know in which eggs it can looking for tests. I have find a bug and fixed it. So now there are a new released of this egg (done with mkrelease)
After lunch I had decided to work on Plone4 buildout to fix coreloadtests.cfg. Because I m new to the community I had done this work under the collective.funkbot repository (still collective but not svn plone).
Next I have used my paster to initialize the buildout of the futur nighly performance bot (still under collective.funkbot repository)
Before the sprint report of the day I have succeed in fixing most of bugs. The only remaining one was on the creation of the plone4 site. That has been a bit funny to report this: “I had succeed except I can’t create a Plone site …”
After this report I have finished and add the ability to collective.recipe.plonesite to create a Plone4 site. I have next discovered a bug with the basic theme of Plone, so I have report it to limi and added the profile of plonetheme.sunbirst to launch funkload.
I have also refresh a bit coreloadtests by moving genericsetup registrations and steps in zcml.
I have also made this report :)
You can check this picture:
TODO
- Push my patch on plone4 buildout
- Refresh the scripts that send the performance reports by email (atm their is a simple python script) and the one who serve html reports throw paste server #static (this one is also used by links inside the email)
- Make a better documentation
- Make contentgenerator faster (it seems not working at all)
Conclusion
I want to thanks my companie Makina-Corpus to have give me the ability to comes to conference and sprint.
I want to thanks the Plone community to be so full of great and impressive people !
I have learn a lot about many different topics (hosting, testing, performances, futur of Plone, …)
I want to be at the next sprints and at the next Plone conferences !
I will continue to work on that topic, so to stay connected, you can follow me on twitter with the nickname toutpt that you can’t pronounce in english, so if you try , do it like “toopatai” :)
Links
- collective.funkbot
- plone4 buildout patched to be triggered by buildout with coreloadtests
- [The nighly buildout buildbot]](http://svn.plone.org/svn/collective/collective.funkbot/plone4funkbotbuildout/) (pasted from funkbot and adapted).