Jaunty Jackelope, compared to the Karmic Koala Beta I recently installed, seems to have been to Canonical what Windows Vista was to Microsoft. And both Jaunty and Vista suffered problems for the same reason; the OS's included too many new and "improved" features that hardware and software vendors just weren't ready for.
What really was the major issue with Jaunty was a newer version of the X server that eschewed the proven track record of using xorg.conf files for the auto-generated X configurations. Great idea but didn't work so well, especially for folks forced to use older graphics drivers or even anyone using ATI graphics cards and ATI's proprietary drivers. For myself, with an ATI card installed at home, it meant I had to do some rather tedious work arounds to get my graphics card working on Jaunty.
Another issue was the switch to Pulse audio being the default sound sub-system in Jaunty. Again, Pulse is great technology but people just weren't fully prepared for it. To this day I still have sound issues with my USB sound card "dongle" for my headphones, as well as issues with sound from Flash video in Firefox.
With Karmic, it looks as if the Ubuntu developers have not messed around with systems so dependant on outside support such as the graphics and sound systems. Massive improvements to the startup systems though have meant I have boot times of around 5 to 10 seconds (yes you read that right).
Jaunty was, however never an out and out disaster like Vista was for Microsoft. It had its issues, but with a 6 month release cycle, as opposed to Microsoft's years of development, things can get back on track very quickly, and Karmic sure seems to show that they are.
One of the big advantages of agile, open source development that Canonical has over Microsoft.
Monday, October 26, 2009
Wednesday, October 14, 2009
PHP Session write locking and how to deal with it in symfony
It has been a while since I last posted to my blog. My personal life has seen a lot of upheaval recently with a house move, a hiatus from Synaq, then back to Synaq, and the release of Pinpoint 2 to Synaq customers all playing a major role in eating into my personal time.
So as my comeback article I thought I would write on an issue that was plaguing us in development of Pinpoint and how, thanks to help from the great symfony community at the symfony users mailing list, we got it resolved.
The problem
With Pinpoint 2, a symfony based application we have developed here at Synaq, we employ quite a lot of Ajax requests in order to make the interface more responsive and less bandwidth hungry; why reload an entire page when you really only want a small sub-set of that page to change? A problem came about when we had an Ajax request running, and while this request was waiting for a response from the server, if a user clicked another link, that link would not "process" until the previous Ajax request had completed. What this meant to us was that it seemed that our requests were "queuing" instead of working asynchronously as they should.
On one particular section of Pinpoint, we have a number of Ajax requests loading at once, each one interrogating the database for data. Each of those requests "queued" behind each other, and any attempt by the user to go to another module resulted in waiting for each of these queued requests to complete before the browser would process the users interaction.
The cause
After going through all sorts of different possible fixes, none of which worked, I eventually submitted the above problem to the symfony users mailing list. The response that came back was that it probably had something to do with PHP session-write locking.
PHP manages sessions, this anyone who codes in PHP knows, and in order to ensure that session data does not become corrupted between requests, PHP will lock write access to the session files for a user while it is processing a request. This results in the following process if you have multiple requests coming through:
1. Request comes into server, and PHP locks session files.
2. Another request comes in but cannot access the session files because they are locked.
3. The first request processes, running all SQL, processing results, etc.
4. Yet another request comes in but cannot process because session files are locked.
5. The first response is finally finished, sends its output back to the calling function and unlocks session files.
6. The second request begins processing, locks session files and continues to do what it needs to.
7. Request three is still waiting for session access.
8. Yet another request comes in but ..... I think you get the picture.
The solution
The only way to resolve this issue is to force the requests to unlock the session files as soon as possible. Thankfully symfony has its own user session storage classes that make this incredibly easy.
The one problem is that you cannot release the session lock until after you have saved data into session that needs to be saved. Our solution was that for each action that processes an Ajax request, write everything as soon as possible to session that needs it and then unlock session to allow any other request to begin processing.
We hit a roadbump. Using symfony's $this->getUser()->setAttribute() command to store session data, we then used PHP's session_write_close() to force PHP to let go of the lock and let the next request begin work. This did unlock session but we noticed that all the data allocated to session using $this->getUser()->setAttribute() was not saved.
After a little exploration of the symfony classes we noticed that when the setAttribute() method is used, in order to speed up processing, symfony does not immediately write to the global PHP $_SESSION variable. Instead it keeps those values in an array until the end of script execution and only then writes to session. Using PHP's session_write_lock() we pretty much made it impossible for symfony to do this because to prevent session data from losing concurrency, PHP does not allow a script to write to session if the session was unlocked.
We did, however, find another method: $this->getUser()->shutdown(). This forces symfony, when the shutdown() method is called, to write session data into $_SESSION and then it also runs session_write_close() itself.
The end result
We now have actions that process Ajax requests and once all data has been sent to session using $this->getUser()->setAttribute() we run the $this->getUser()->shutdown() method. The difference was incredible and has actually speeded up our entire application a ton.
One thing to be careful of however. You do need to be sure that you call that shutdown() command at the right time, because if you call it too early, session data will not get saved and PHP will just ignore it. We had to reshuffle some code so that all the database calls and data processing functions were run after shutdown() as well.
Thanks again to the symfony community for helping to point this out and hope this helps others who may have the same issue as well.
So as my comeback article I thought I would write on an issue that was plaguing us in development of Pinpoint and how, thanks to help from the great symfony community at the symfony users mailing list, we got it resolved.
The problem
With Pinpoint 2, a symfony based application we have developed here at Synaq, we employ quite a lot of Ajax requests in order to make the interface more responsive and less bandwidth hungry; why reload an entire page when you really only want a small sub-set of that page to change? A problem came about when we had an Ajax request running, and while this request was waiting for a response from the server, if a user clicked another link, that link would not "process" until the previous Ajax request had completed. What this meant to us was that it seemed that our requests were "queuing" instead of working asynchronously as they should.
On one particular section of Pinpoint, we have a number of Ajax requests loading at once, each one interrogating the database for data. Each of those requests "queued" behind each other, and any attempt by the user to go to another module resulted in waiting for each of these queued requests to complete before the browser would process the users interaction.
The cause
After going through all sorts of different possible fixes, none of which worked, I eventually submitted the above problem to the symfony users mailing list. The response that came back was that it probably had something to do with PHP session-write locking.
PHP manages sessions, this anyone who codes in PHP knows, and in order to ensure that session data does not become corrupted between requests, PHP will lock write access to the session files for a user while it is processing a request. This results in the following process if you have multiple requests coming through:
1. Request comes into server, and PHP locks session files.
2. Another request comes in but cannot access the session files because they are locked.
3. The first request processes, running all SQL, processing results, etc.
4. Yet another request comes in but cannot process because session files are locked.
5. The first response is finally finished, sends its output back to the calling function and unlocks session files.
6. The second request begins processing, locks session files and continues to do what it needs to.
7. Request three is still waiting for session access.
8. Yet another request comes in but ..... I think you get the picture.
The solution
The only way to resolve this issue is to force the requests to unlock the session files as soon as possible. Thankfully symfony has its own user session storage classes that make this incredibly easy.
The one problem is that you cannot release the session lock until after you have saved data into session that needs to be saved. Our solution was that for each action that processes an Ajax request, write everything as soon as possible to session that needs it and then unlock session to allow any other request to begin processing.
We hit a roadbump. Using symfony's $this->getUser()->setAttribute() command to store session data, we then used PHP's session_write_close() to force PHP to let go of the lock and let the next request begin work. This did unlock session but we noticed that all the data allocated to session using $this->getUser()->setAttribute() was not saved.
After a little exploration of the symfony classes we noticed that when the setAttribute() method is used, in order to speed up processing, symfony does not immediately write to the global PHP $_SESSION variable. Instead it keeps those values in an array until the end of script execution and only then writes to session. Using PHP's session_write_lock() we pretty much made it impossible for symfony to do this because to prevent session data from losing concurrency, PHP does not allow a script to write to session if the session was unlocked.
We did, however, find another method: $this->getUser()->shutdown(). This forces symfony, when the shutdown() method is called, to write session data into $_SESSION and then it also runs session_write_close() itself.
The end result
We now have actions that process Ajax requests and once all data has been sent to session using $this->getUser()->setAttribute() we run the $this->getUser()->shutdown() method. The difference was incredible and has actually speeded up our entire application a ton.
One thing to be careful of however. You do need to be sure that you call that shutdown() command at the right time, because if you call it too early, session data will not get saved and PHP will just ignore it. We had to reshuffle some code so that all the database calls and data processing functions were run after shutdown() as well.
Thanks again to the symfony community for helping to point this out and hope this helps others who may have the same issue as well.
Subscribe to:
Posts (Atom)