Re: Session and Multi Server Architecture

Per Jessen wrote:

Stut wrote:

Per Jessen wrote:

mike wrote:

Check out persistency in LVS for instance: http://www.linuxvirtualserver.org/docs/persistence.html

i know persistence handling is an option in LVS, but i haven't seen the need to use it. i use LVS right now without even bothering with any of that.

Because you've chosen another option - memcached presumably - which is more expensive over all. (IMHO).

Based on what? I'm currently looking at replacing a file-based cache with memcached and I'm interested in all justified opinions.

You might not have much use for mine then :-) If I compare plain session-persistency (session being a client to server relationship, not $_SESSION) with memcacheds copying objects around across many machines, common sense tells me that the latter will use a lot more resources. Both on the network and on each machine. With memcached, and 1000 clients for a cluster of 50 machines, each using up 1Mb of $_SESSION space, that is 1000Mb per machine, 50Gb in total. With session-persistency, you have 50 machines with 1000/50 clients each, making 20Mb per machine. And virtually no session-related inter-machine network traffic. And your machines are not busy with keeping 980 objects needlessly up-to-date. Of course, processing power, network capacity and memory are all very cheap these days, so it's easy to put on the Microsoft hat and be wasteful.

In my mind you're exchanging traffic over a local network (probably 1Gbps) for a less resilient load balancing system. By locking users to particular machines you don't allow for the possibility that you'll end up with a large number of users on any given server while others lie idle. Personally I'd always opt for a solution where my hardware will be utilised as evenly as possible regardless of user patterns.

On the next request, LVS will know not to try that server, and the user will move to another one. Obviously the session-context will die, but is that really a big deal? How often does one of your servers die? Sure, if you've got 10,000 in a cluster, you'll have fans and harddisks pop every so often, but I have my doubts about memcached scaling to that level (please correct me if I'm wrong here, I have _no_ experience with memcached).

Facebook. Digg. LiveJournal. That enough scalability proof for ya? If not there are plenty more where those came from.

It wasn't really the scalability I was interested in here, but I'd be interested to hear some numbers if you have any.

Mike posted a link to a thread that discusses Facebook usage. I don't have any other numbers handy but I'm sure Google can find some for you if needed.

My point with the 10,000 machines was - with that many components, even the 100,000 hours MTBF of a regular harddisk will mean one will break every ten hours (statistically speaking). Add fan- and other failures to that, and you'll have sessions dying quite frequently. Which _could_ be a significant problem, and maybe enough to make you want to use memcached for object replication. (there might also be a restriction in the number of concurrent clients handled by LVS session persistency). However, with a much smaller cluster, say 50 or 100 machines, hardware failures will be a lot less frequent, and having the odd session die once a week might not be a problem. Which is why I think LVS session persistency is perfectly fine.

Which is why memcache should never be the definitive source for any data. Memcache is volatile storage, that's a fact you can't get away from. If you're using it to store session data then you either accept that sessions might die at any time or you back it up with storage in something more persistent. I'll be using memcache as a simple cache. I hate sessions and avoid them for anything but the most trivial sites. The main sites I work with no longer use sessions because they add a pointless layer of complexity to any application that need to scale beyond a single machine. Maybe I'm just lucky that the applications I develop don't need to store huge amounts of temporary data during a visit, but I'm yet to come across a reason to use "session data". -Stut -- http://stut.net/