| --- | Log | opened Wed Jul 23 00:00:25 2008 |
| 00:03 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has quit [Remote host closed the connection] |
| 00:04 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has joined #linode |
| 00:06 | -!- | Kassah [~kassah@24-116-150-254.cpe.cableone.net] has joined #linode |
| 00:09 | -!- | t3mp3st [~t3mp3st@pool-71-125-56-177.nycmny.fios.verizon.net] has joined #linode |
| 00:09 | <t3mp3st> | Hi all; I'm looking forward to starting a linode account as was hoping someone might be able to answer a few simple and, in all likelihood, dumb questions :) |
| 00:11 | <StevenK> | t3mp3st: Just ask :-) |
| 00:11 | <t3mp3st> | StevenK: thanks :) |
| 00:12 | <t3mp3st> | one of my goals is scalability; however, it seems a little tricky "doing it right" -- especially since we have total freedom in setting things up |
| 00:12 | <t3mp3st> | in particular, I'd like to set up a "cluster" of images, perhaps having one serve static media |
| 00:12 | <t3mp3st> | two or three serve dynamic content, and perhaps another to host a mysql database |
| 00:13 | <t3mp3st> | scaling beyond this, however, is a bit of a mystery; I haven't been able to find any documentation on the wiki describing a nice, easy-to-scale setup and was wondering if you (or anyone else) might have any experience with this sort of thing |
| 00:13 | <Peng_> | That's more of a general Linux/Unix question. |
| 00:14 | <t3mp3st> | well, I was wondering if there was a "common case" for achieving this sort of thing at Linode |
| 00:14 | <t3mp3st> | and if not, I'd be really grateful for any pointers you could offer to documents that might help to better understand how to set something like this up effectively |
| 00:15 | <Peng_> | I don't know if there's a common setup, and I can't really help. |
| 00:15 | <Syn-> | there's many ways of doing that tbh |
| 00:15 | <t3mp3st> | hmm |
| 00:15 | <Syn-> | from having a File server running NFS hosting content that is mounted on several web servers |
| 00:15 | <Peng_> | BTw, Linode will be beta-ing an NAS service soon. |
| 00:15 | <Syn-> | and using lvm for load balancers (tho hardware is much better ofc) |
| 00:16 | <Syn-> | and various implementations of that setup |
| 00:16 | <Syn-> | to going all out sharding data at a datacenter and global level, something im starting into at work |
| 00:16 | <Syn-> | http://www.highscalability.com/ |
| 00:17 | <Syn-> | decent site with insight and backgrounds on a lot of large and small sites that are focused on scaling |
| 00:17 | <t3mp3st> | do they talk a bit about the "how", or is this mostly for folks who already know :P? |
| 00:17 | <Syn-> | both |
| 00:18 | <Syn-> | youll have to do a ton of reading |
| 00:18 | <Syn-> | but if you dont know how to do basic Unix things (like NFS concepts) you will get lost pretty quick |
| 00:19 | <Syn-> | memcached, lighttpd, and varnish are the hot daemons of late in helping sites get better performance on each box AND help the overall cluster scale |
| 00:19 | <Syn-> | as you'll read, it's not about vertical scaling or horizontal, its about the diagonal scaling you want to aim for - using linodes is probably a good way to test that out |
| 00:20 | <Peng_> | Nginx too |
| 00:20 | <t3mp3st> | I'm familiar with NFS-- I've just never really had an excuse to play with clustering until now :P |
| 00:20 | <Syn-> | yea nginx's lack of english docs (until recently I believe) hasnt made me very keen on it |
| 00:20 | <t3mp3st> | I guess it's time to drink from the firehose |
| 00:20 | <Syn-> | but it and lighttpd are apache killers |
| 00:21 | <t3mp3st> | i've stumbled upon the two-- |
| 00:21 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has quit [Remote host closed the connection] |
| 00:21 | <t3mp3st> | but can they be used to serve dynamic content? |
| 00:21 | <Syn-> | i have a 100mio site on lighttpd right now, and im looking at using lighttpd in all of our future global infrastructure |
| 00:21 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has joined #linode |
| 00:21 | <Syn-> | yeah |
| 00:22 | <Syn-> | php wise definitely |
| 00:22 | <t3mp3st> | cool |
| 00:22 | <jackc> | have fun |
| 00:22 | <t3mp3st> | (I've got a Django app) |
| 00:22 | <Syn-> | i dont have any RoR experience, but they both use fastcgi for php (vs mod_php in apache) |
| 00:22 | <StevenK> | I use mod_rails for my RoR app |
| 00:23 | -!- | JDLSpeedy [~joe@fl-69-34-4-219.sta.embarqhsd.net] has quit [Ping timeout: 480 seconds] |
| 00:26 | <Peng_> | Oh, right, JDLSpeedy and I have the same ISP. I thought the other me had pinged out or something. |
| 00:30 | -!- | cyco [~4a04ea27@webuser.linode.com] has quit [Remote host closed the connection] |
| 00:35 | -!- | internat [~nf@123-243-184-161.static.tpgi.com.au] has quit [Read error: Connection reset by peer] |
| 00:36 | -!- | internat [~nf@123-243-184-161.static.tpgi.com.au] has joined #linode |
| 00:40 | -!- | vs9 [~Josh_Wine@dyn216-8-131-77.ADSL.mnsi.net] has joined #linode |
| 00:40 | -!- | vs9 [~Josh_Wine@dyn216-8-131-77.ADSL.mnsi.net] has quit [] |
| 00:41 | -!- | vs9 [~Josh_Wine@dyn216-8-131-77.ADSL.mnsi.net] has joined #linode |
| 00:41 | -!- | vs9 [~Josh_Wine@dyn216-8-131-77.ADSL.mnsi.net] has quit [] |
| 00:46 | -!- | vs9 [~Josh_Wine@dyn216-8-131-77.ADSL.mnsi.net] has joined #linode |
| 00:46 | -!- | vs9 [~Josh_Wine@dyn216-8-131-77.ADSL.mnsi.net] has quit [] |
| 00:49 | -!- | SelfishMan [~TheSelfis@host-72-175-55-117.lvt-mt.client.bresnan.net] has joined #linode |
| 00:50 | -!- | TheFirst [gaveup@your.friendly.neighborhood.hellmouth.info] has quit [Ping timeout: 480 seconds] |
| 00:53 | -!- | arooni-mobile [~arooni-mo@c-67-168-26-198.hsd1.wa.comcast.net] has joined #linode |
| 00:55 | <t3mp3st> | hey-- any of you folks have any good book recommendations for learning about building high availability Linux clusters? |
| 01:03 | -!- | JDLSpeedy [~joe@fl-69-34-4-219.sta.embarqhsd.net] has joined #linode |
| 01:08 | -!- | arooni-mobile__ [~arooni-mo@c-67-168-26-198.hsd1.wa.comcast.net] has joined #linode |
| 01:11 | -!- | arooni-mobile [~arooni-mo@c-67-168-26-198.hsd1.wa.comcast.net] has quit [Ping timeout: 480 seconds] |
| 01:13 | -!- | bliblok [~bjornar@ti500720a080-8908.bb.online.no] has quit [Read error: Connection reset by peer] |
| 01:13 | -!- | bliblok [~bjornar@ti500720a080-8908.bb.online.no] has joined #linode |
| 01:29 | -!- | r3z` [~r3z@c-68-58-96-186.hsd1.in.comcast.net] has joined #linode |
| 01:31 | <SelfishMan> | !define coprophobia |
| 01:31 | -!- | arooni-mobile__ [~arooni-mo@c-67-168-26-198.hsd1.wa.comcast.net] has quit [Ping timeout: 480 seconds] |
| 01:31 | <SelfishMan> | Um....what the hell |
| 01:31 | <SelfishMan> | Seriously, who would use that as a password? |
| 01:33 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has quit [Remote host closed the connection] |
| 01:33 | <ijustam> | heh |
| 01:33 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has joined #linode |
| 01:35 | -!- | r3z`` [~r3z@c-68-58-96-186.hsd1.in.comcast.net] has quit [Ping timeout: 480 seconds] |
| 01:46 | -!- | queso|away [~queso@cpe-71-79-6-95.cinci.res.rr.com] has quit [Quit: Leaving...] |
| 01:57 | -!- | SelfishMan [~TheSelfis@host-72-175-55-117.lvt-mt.client.bresnan.net] has quit [Read error: Connection reset by peer] |
| 02:04 | -!- | ijustam [~ija@c-68-58-59-13.hsd1.in.comcast.net] has quit [Quit: Leaving] |
| 02:17 | -!- | kupesoft [~dave@CPE004314674170-CM0018c0c44e76.cpe.net.cable.rogers.com] has quit [Quit: Ex-Chat] |
| 02:20 | -!- | Noia [~none@host-84-9-196-31.dslgb.com] has joined #linode |
| 02:27 | -!- | Midicronica [~none@host-84-9-196-31.dslgb.com] has quit [Ping timeout: 480 seconds] |
| 02:36 | -!- | t3mp3st [~t3mp3st@pool-71-125-56-177.nycmny.fios.verizon.net] has quit [Quit: Leaving] |
| 02:41 | -!- | SelfishMan [~TheSelfis@host-72-175-55-117.lvt-mt.client.bresnan.net] has joined #linode |
| 02:46 | -!- | arooni-mobile [~arooni-mo@c-67-168-26-198.hsd1.wa.comcast.net] has joined #linode |
| 03:02 | -!- | metaperl [~metaperl@cpe-75-187-102-204.insight.res.rr.com] has quit [Ping timeout: 480 seconds] |
| 03:02 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has quit [Remote host closed the connection] |
| 03:03 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has joined #linode |
| 03:03 | -!- | metaperl [~metaperl@cpe-75-187-102-204.insight.res.rr.com] has joined #linode |
| 03:12 | -!- | giri [~giri@adsl-76-225-158-71.dsl.pltn13.sbcglobal.net] has joined #linode |
| 03:12 | <giri> | hey guys |
| 03:12 | <giri> | how can i check if postfix is running chrooted? |
| 03:13 | <giri> | i forgot if i did that during setup... |
| 03:14 | <StevenK> | It's in /etc/postfix/master.cf |
| 03:15 | <StevenK> | smtp inet n - - - - smtpd |
| 03:15 | <StevenK> | That second dash is 'run chrooted' |
| 03:16 | <giri> | ok |
| 03:16 | <giri> | so if there is a dash, it is running chrooted |
| 03:17 | <StevenK> | Right |
| 03:17 | <giri> | i also did 'cd ~postfix' and it took me to /var/spool/postfix |
| 03:17 | <giri> | having an issue getting smtp-auth working with authdaemon |
| 03:17 | <StevenK> | Since that's the home directory of the user postfix |
| 03:17 | <giri> | looks like i need to move authdamon to where postfix can see it |
| 03:17 | -!- | TehDan [~dan@zoidberg.area-52.org.uk] has joined #linode |
| 03:17 | <giri> | will this break courier tho? |
| 03:18 | <StevenK> | I have dovecot exporting an auth socket to /var/spool/postfix/private/auth |
| 03:18 | <StevenK> | And postfix configured to ask private/auth |
| 03:19 | <giri> | i c |
| 03:19 | <giri> | can courier/authdaemon do that? |
| 03:19 | <giri> | i've been googling around and seeing ppl create symlinks |
| 03:20 | <StevenK> | I don't think symlinks will work |
| 03:20 | <StevenK> | In terms of courier/authdaemon, I have no idea |
| 03:22 | <TehDan> | is dalas having problems? |
| 03:24 | <giri> | i'm working on dallas80, no problems |
| 03:25 | <TehDan> | looks like a routing issue with my ISP... |
| 03:27 | -!- | ryan8403 [~ryan8403@home.ryanchewning.net] has joined #linode |
| 03:27 | -!- | ryan8403 [~ryan8403@home.ryanchewning.net] has left #linode [] |
| 03:48 | <giri> | i tried editing the authdaemon init.d scripts |
| 03:48 | <giri> | to create hard links |
| 03:48 | <giri> | ln: creating hard link `./pid' => `/var/run/courier/authdaemon/pid': Invalid cross-device link |
| 03:48 | <giri> | getting that error |
| 03:48 | <giri> | when i try to stop/start |
| 03:50 | <giri> | maybe i should step back, see if there's another solution |
| 03:50 | <StevenK> | /var/run is a tmpfs |
| 03:50 | <giri> | so i'm trying to enable smtp authorization |
| 03:50 | <StevenK> | And hard links can't jump filesystems |
| 03:51 | <giri> | ok |
| 03:51 | <giri> | so i have postfix/courier/mysql/postfixadmin |
| 03:51 | <giri> | after some wrangling, i have everything using a shared DB |
| 03:51 | <giri> | and in the DB i have encrypted passwords |
| 03:52 | <giri> | smtp auth uses sasl, which can go to the DB or use authdaemon (other methods?) |
| 03:52 | <StevenK> | I suspect part of the solution is to run smtpd non chrooted |
| 03:52 | <giri> | yeah that would probably be the easiest fix |
| 03:52 | <giri> | do i have to wory about other problems if i do that? |
| 03:54 | <StevenK> | Just thinking about it, you could configure saslauthd to export a socket into the /var/spool/postfix chroot base, and have saslauthd to talk to your database |
| 03:56 | <giri> | yeah i can do that. another service to run (saslauthd) |
| 03:56 | <giri> | a backup option |
| 03:58 | -!- | spasmface [~spasm@r220-101-68-180.cpe.unwired.net.au] has quit [Quit: Leaving...] |
| 04:00 | <StevenK> | saslauthd is ... odd, though |
| 04:03 | <giri> | yeah. i give up on smtp via authdaemond |
| 04:03 | <giri> | no one has been able to make it work. |
| 04:03 | <giri> | spent 6 hrs on it. that's enough. using saslauthd |
| 04:03 | -!- | getsmart [~getsmart@88-149-241-182.dynamic.ngi.it] has joined #linode |
| 04:06 | -!- | alexyur [~5b7af54d@webuser.linode.com] has joined #linode |
| 04:07 | <alexyur> | Hello. My linode is to responsible. Can anybody help? |
| 04:07 | <alexyur> | Hello. My linode is NOT responsible. Can anybody help |
| 04:08 | <alexyur> | What to do? |
| 04:08 | <StevenK> | Log in using lish |
| 04:09 | <alexyur> | I ordered to reboot it. That job in queue, and i cant log into this linode |
| 04:09 | <alexyur> | In queue very long, about 30 minutes |
| 04:10 | <StevenK> | Are you able to log into the lish shell? |
| 04:11 | <alexyur> | through ssh? |
| 04:12 | <StevenK> | !lish |
| 04:12 | <@linbot> | StevenK: "lish" could be allows you to perform certain actions without having to log into the LPM. Lish's primary function is to allow you access to your server's console, even if networking is disabled. http://www.linode.com/wiki/index.php/Lish_Documentation |
| 04:12 | -!- | jcr [~63ee1192@webuser.linode.com] has joined #linode |
| 04:12 | <StevenK> | alexyur: ^ |
| 04:13 | <jcr> | hello, I am tying to reboot a linode but clicking on "reboot" but the jobs are still in the queue after 10 minutes |
| 04:13 | <alexyur> | https://www.linode.com/members/ -> Linode manager -> console, here? |
| 04:13 | <StevenK> | I wonder if alexyur and jcr are on the same host |
| 04:13 | <alexyur> | jcr, i have the same problem |
| 04:14 | <jcr> | I am on newark21 |
| 04:14 | <encode> | have either of you raised a support ticket? |
| 04:14 | <alexyur> | I am on newark5.linode.com |
| 04:14 | <alexyur> | Yes |
| 04:14 | <StevenK> | I would be curious if either of you, jcr and alexyur could connect to lish |
| 04:16 | <alexyur> | I can connect to newark5.linode.com via port 22, but it is not my virtual machine, something else |
| 04:17 | <encode> | alexyur: it is the host that runs your virtual machine |
| 04:17 | <encode> | try logging in with the same credentials you use to login to the web interface |
| 04:18 | <jcr> | the shutdown finally completed a few seconds ago |
| 04:19 | <jcr> | and my attemps to connect with lish failed with the wronf login |
| 04:19 | <alexyur> | It seems it waked up. |
| 04:19 | <SelfishMan> | Sounds to me like the nodes crashed and took a full 2.5 minutes to reboot |
| 04:20 | <SelfishMan> | Sometimes it does take longer than 2.5 minutes unfortunately |
| 04:20 | <alexyur> | Over 30 minutes for me |
| 04:21 | <encode> | probably because all the nodes on that host were rebooting at once |
| 04:21 | <encode> | what plan are you on? /me bets a linode 360 |
| 04:21 | <jcr> | yes on a 360 and the rebooting takes also a lot of time |
| 04:21 | -!- | besonen_mobile [~besonen_m@71-220-224-216.eugn.qwest.net] has joined #linode |
| 04:23 | <alexyur> | I thouugt that the problem in Linux installation, but my another Linux boots also very long |
| 04:24 | <tozz> | 30minutes to boot seems extreme, even when newart12 went down yesterday my 360 went up in just a few mins |
| 04:24 | <@linbot> | New news from forums: several crontab files in Linux, Apache, Mysql and PHP (LAMP) Forum <http://www.linode.com/forums/viewtopic.php?t=3399>; |
| 04:25 | <giri> | damn i'm still not able to get smtp-auth working |
| 04:25 | * | StevenK kicks SpaceHobo in the roll |
| 04:26 | <jcr> | my "system boot" job is finally in progress |
| 04:26 | <SelfishMan> | My nodes boot in less than a minute, typically less than thirty seconds. If it is taking more than 5 minutes then you are thrashing the disk or something similar. |
| 04:30 | <SelfishMan> | I don't get it |
| 04:30 | <SelfishMan> | I still don't get it |
| 04:31 | <SelfishMan> | Huh? |
| 04:31 | * | SelfishMan wonders how long he can keep SpaceHobo doing this |
| 04:31 | <@mikegrb> | lolz |
| 04:31 | <SelfishMan> | lol |
| 04:31 | <jcr> | my linode in newark21 was finally able to reboot |
| 04:31 | <jcr> | all is good now |
| 04:32 | <SelfishMan> | That's the same as the other one |
| 04:32 | -!- | tsp [~tyler@S0106001310788ff0.vf.shawcable.net] has joined #linode |
| 04:32 | -!- | Tallo [~meer@rigel.nutomix.com] has joined #linode |
| 04:37 | -!- | giri [~giri@adsl-76-225-158-71.dsl.pltn13.sbcglobal.net] has quit [Remote host closed the connection] |
| 04:42 | -!- | ondrej [~ondra@ip4-83-240-41-73.cust.nbox.cz] has joined #linode |
| 04:45 | -!- | jcr [~63ee1192@webuser.linode.com] has quit [Remote host closed the connection] |
| 04:45 | -!- | getsmart [~getsmart@88-149-241-182.dynamic.ngi.it] has quit [Quit: Ex-Chat] |
| 05:06 | -!- | jimcooncat [~jim@pool-72-65-110-79.ptldme.east.verizon.net] has joined #linode |
| 05:39 | -!- | ryan8403 [~ryan8403@home.ryanchewning.net] has joined #linode |
| 05:48 | -!- | JDLSpeedy [~joe@fl-69-34-4-219.sta.embarqhsd.net] has quit [Ping timeout: 480 seconds] |
| 06:00 | -!- | spasmface [~spasm@r220-101-68-180.cpe.unwired.net.au] has joined #linode |
| 06:26 | -!- | JDLSpeedy [~joe@fl-69-34-4-219.sta.embarqhsd.net] has joined #linode |
| 06:34 | -!- | internat1 [~nf@123-243-184-161.static.tpgi.com.au] has joined #linode |
| 06:34 | -!- | internat [~nf@123-243-184-161.static.tpgi.com.au] has quit [Read error: Connection reset by peer] |
| 06:41 | -!- | ondrej [~ondra@ip4-83-240-41-73.cust.nbox.cz] has quit [Remote host closed the connection] |
| 06:51 | -!- | JDLSpeedy [~joe@fl-69-34-4-219.sta.embarqhsd.net] has quit [Quit: Client exiting] |
| 06:57 | -!- | MrGeneral [~MrGeneral@bl7-110-22.dsl.telepac.pt] has quit [Ping timeout: 480 seconds] |
| 06:57 | -!- | MrGeneral [~MrGeneral@bl7-173-121.dsl.telepac.pt] has joined #linode |
| 06:58 | -!- | JDLSpeedy [~joe@fl-69-34-4-219.sta.embarqhsd.net] has joined #linode |
| 07:01 | -!- | binel [~h00s@83-131-77-148.adsl.net.t-com.hr] has joined #linode |
| 07:07 | -!- | binel_ [~h00s@78-0-242-17.adsl.net.t-com.hr] has quit [Ping timeout: 480 seconds] |
| 07:17 | -!- | digx [~rick@c-76-109-201-140.hsd1.fl.comcast.net] has joined #linode |
| 07:19 | -!- | euph [~da@nh80.nathist.au.dk] has quit [Server closed connection] |
| 07:20 | -!- | euph [~da@nh80.nathist.au.dk] has joined #linode |
| 07:23 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has quit [Remote host closed the connection] |
| 07:24 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has joined #linode |
| 07:24 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe06:ad18] has quit [Remote host closed the connection] |
| 07:24 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe1b:abd3] has joined #linode |
| 07:25 | -!- | Adam12 [4c0aad12@67.207.141.120] has quit [Quit: http://www.mibbit.com ajax IRC Client] |
| 07:26 | -!- | Sputnik9 [~Sputnik7@c-71-192-5-190.hsd1.ma.comcast.net] has quit [Quit: -=SysReset 2.53=-] |
| 07:28 | -!- | Sputnik7 [~Sputnik7@c-71-192-5-190.hsd1.ma.comcast.net] has joined #linode |
| 07:28 | -!- | mendel [banana@76-10-159-31.dsl.teksavvy.com] has quit [Quit: Leaving.] |
| 07:33 | -!- | TheFirst [gaveup@your.friendly.neighborhood.hellmouth.info] has joined #linode |
| 07:37 | -!- | daMaestro [~jon@content.beatport.com] has joined #linode |
| 07:41 | -!- | xitology [~xi@golovko1.donbass.com] has joined #linode |
| 07:46 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has quit [Ping timeout: 480 seconds] |
| 07:47 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has joined #linode |
| 08:02 | -!- | TheFirst [gaveup@your.friendly.neighborhood.hellmouth.info] has quit [Ping timeout: 480 seconds] |
| 08:07 | -!- | mendel [~rich@TOROON12-1279379717.sdsl.bell.ca] has joined #linode |
| 08:10 | -!- | digx [~rick@c-76-109-201-140.hsd1.fl.comcast.net] has quit [Quit: digx] |
| 08:28 | -!- | Tallo [~meer@rigel.nutomix.com] has quit [Ping timeout: 480 seconds] |
| 08:50 | -!- | getsmart [~getsmart@88-149-241-182.dynamic.ngi.it] has joined #linode |
| 08:55 | -!- | getsmart [~getsmart@88-149-241-182.dynamic.ngi.it] has quit [Quit: Ex-Chat] |
| 09:23 | -!- | alexyur [~5b7af54d@webuser.linode.com] has quit [Remote host closed the connection] |
| 09:25 | -!- | jcr [~63ee1192@webuser.linode.com] has joined #linode |
| 09:26 | <jcr> | does anyone experience massive slowness on a newark21 machine? |
| 09:27 | <jcr> | I have restarted the server and right away, uptime takes a while to gives number like 7. I had to restart because uptime was showing numbers above 30 |
| 09:30 | -!- | simlun [~simon@84-217-126-0.tn.glocalnet.net] has joined #linode |
| 09:30 | * | mikegrb takes a look |
| 09:31 | <jcr> | top indicates that I have 6 zombies; looks like a similar problem to earlier this morning, or maybe to what is described here http://www.linode.com/forums/viewtopic.php?t=3385 |
| 09:32 | <m0> | We need a good peering UK Server :) |
| 09:32 | -!- | daMaestro [~jon@content.beatport.com] has quit [Quit: Leaving] |
| 09:32 | -!- | daMaestro [~jon@content.beatport.com] has joined #linode |
| 09:32 | -!- | RiverRat [me@97-112-149-253.clsp.qwest.net] has quit [Ping timeout: 480 seconds] |
| 09:34 | -!- | simlun1 [~simon@84-217-119-80.tn.glocalnet.net] has quit [Ping timeout: 480 seconds] |
| 09:34 | <@caker> | jcr: how's it looking now? |
| 09:37 | <jcr> | much better over the last minutes |
| 09:38 | <jcr> | uptime is back at 1 and web pages are responive again |
| 09:38 | <Peng_> | caker: What was going on? |
| 09:38 | <@caker> | thrasher. |
| 09:38 | <jcr> | no more zombies also |
| 09:38 | <@caker> | idle cfq class he goes |
| 09:39 | <Peng_> | Since Xen doesn't have I/O tokens, are thrashers more of a problem? |
| 09:39 | <@mikegrb> | just slightly |
| 09:39 | -!- | digx [~rick@131.94.183.248] has joined #linode |
| 09:40 | <BP{k}> | hmm linodes in the uk .. I like that idea ;) |
| 09:40 | <Peng_> | mikegrb: This one was pretty serious. They were almost this bad on UML? |
| 09:45 | -!- | queso [~queso@cpe-71-79-6-95.cinci.res.rr.com] has joined #linode |
| 09:45 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has quit [Ping timeout: 480 seconds] |
| 09:49 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has joined #linode |
| 10:01 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has quit [Ping timeout: 480 seconds] |
| 10:03 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has joined #linode |
| 10:03 | -!- | jm [~fake@c-76-113-194-7.hsd1.mn.comcast.net] has joined #linode |
| 10:03 | -!- | simlun [~simon@84-217-126-0.tn.glocalnet.net] has quit [Ping timeout: 480 seconds] |
| 10:05 | -!- | simlun [~simon@84-217-123-22.tn.glocalnet.net] has joined #linode |
| 10:08 | -!- | simlun [~simon@84-217-123-22.tn.glocalnet.net] has quit [] |
| 10:11 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has quit [Ping timeout: 480 seconds] |
| 10:12 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has joined #linode |
| 10:13 | -!- | michiel_ [~michiel@212.248.241.213] has quit [Quit: leaving] |
| 10:15 | -!- | bkirkbri [~7a1d4975@webuser.linode.com] has joined #linode |
| 10:16 | -!- | ryan8403 [~ryan8403@home.ryanchewning.net] has quit [Quit: Leaving.] |
| 10:17 | -!- | Darxus [~darxus@panic.chaosreigns.com] has left #linode [] |
| 10:21 | -!- | bliblok_ [~bjornar@ti500720a080-8908.bb.online.no] has joined #linode |
| 10:22 | -!- | bliblok [~bjornar@ti500720a080-8908.bb.online.no] has quit [Read error: Connection reset by peer] |
| 10:24 | -!- | TimothyA [~gmtyjk65k@sub-10ip31.onenet.an] has quit [Ping timeout: 480 seconds] |
| 10:26 | -!- | jm [~fake@c-76-113-194-7.hsd1.mn.comcast.net] has quit [] |
| 10:27 | -!- | TimothyA [~gmtyjk65k@sub-216ip242.onenet.an] has joined #linode |
| 10:30 | -!- | queso [~queso@cpe-71-79-6-95.cinci.res.rr.com] has quit [Quit: Leaving...] |
| 10:30 | <@linbot> | New news from forums: Backup MX in Feature Request/Bug Report <http://www.linode.com/forums/viewtopic.php?t=3400>; |
| 10:35 | <HoopyCat> | caker: anything funky on atlanta40? i suspect my I/O is uberslow, as postgres is takin' a looong time |
| 10:36 | -!- | jm [~fake@c-76-113-194-7.hsd1.mn.comcast.net] has joined #linode |
| 10:36 | * | mikegrb hacks HoopyCat's gibson |
| 10:36 | <HoopyCat> | i may be the "funky" :-) |
| 10:38 | * | CaptObviousman hacks HoopyCat's cyberbrain |
| 10:38 | <@mikegrb> | HoopyCat: you are hitting your second swap pretty hard io wise at the moment |
| 10:38 | -!- | lakin [~lakin@S01060013101832cd.cg.shawcable.net] has joined #linode |
| 10:38 | <@mikegrb> | Device: tps blk read/s blk wrt/s blk read blk wrt |
| 10:38 | <@mikegrb> | rocwiki (71902) 459.84 537.86 7094.51 5384 71016 |
| 10:39 | <Peng_> | mikegrb: What command tells you that? |
| 10:39 | <@mikegrb> | Peng_: sexy perl script I wrote |
| 10:39 | <Peng_> | Ah. |
| 10:39 | <@mikegrb> | <3 |
| 10:39 | <@mikegrb> | uses iostat for the real work |
| 10:40 | <@mikegrb> | but then translates device mapper id to linode username + image id, sorts the devices and prints the top 10 |
| 10:40 | <bkirkbri> | mikegrb: can you run that sexy lil' script on newark21? |
| 10:40 | <HoopyCat> | mikegrb: at that particular moment, two of the index.scgi daemons were aiming towards spiraling death; going to write an autoslayer for that particular problem today for sure. |
| 10:40 | <HoopyCat> | mikegrb: things look "normal" right now, heehee |
| 10:40 | <@mikegrb> | bkirkbri: yeah, just a moment |
| 10:41 | <bkirkbri> | mikegrb: thanks! |
| 10:41 | <@mikegrb> | HoopyCat: there was another linode on the host hitting swap even harder, decreased priority for that device a touch |
| 10:41 | <@mikegrb> | should help you out a bit too |
| 10:43 | <HoopyCat> | mikegrb: nod... down from ~12 postgres processes processing to maybe one or two. cool. my application deals with I/O slowness by creating more processes, which is... uhh... something i have to have a word with it about |
| 10:43 | * | mikegrb nods |
| 10:43 | <HoopyCat> | i dropped swap from 256mb to 128mb to increase the chances of OOMing when that happens |
| 10:45 | <@mikegrb> | bkirkbri: you have a linode on n21? |
| 10:45 | <Peng_> | How trustworthy is the OOM killer? Will it always kill whatever's using all the RAM, or will it decide to whack my text editor or something? |
| 10:45 | <bkirkbri> | mikegrb: think so, let me check for sure |
| 10:46 | <HoopyCat> | Peng_: i set panic_on_oom, which cuts down on the uncertainness about what dies |
| 10:46 | <@mikegrb> | Peng_: it does get things wrong from time to time |
| 10:46 | <bkirkbri> | mikegrb: yeah, i have linode10297 on newark21.linode.com |
| 10:46 | <@mikegrb> | k |
| 10:47 | <bkirkbri> | i had a ticket in for SLOOOOOOW i/o last week and tasaro said to let you know if it happened again |
| 10:47 | <bkirkbri> | bing bong, it's happening big time |
| 10:49 | <@mikegrb> | starting to improve? |
| 10:49 | <arachnid> | Anyone know if posix mutexes and condition variables have significant overhead? |
| 10:49 | <arachnid> | Eg, I'm writing a FUSE FS, and pondering having a mutex on every file. Is this bad for large numbers of files? |
| 10:49 | <bkirkbri> | a little bit in the past mintue or so |
| 10:50 | * | mikegrb nods |
| 10:50 | <bkirkbri> | s/tue/ute/ |
| 10:51 | <bkirkbri> | is there some way to limit the thrashing node or is this likely to happen again? |
| 10:51 | <@mikegrb> | limiting is what I did |
| 10:51 | <@mikegrb> | chris limited this one a bit this morning, cranked the screws in some more |
| 10:51 | <bkirkbri> | mikegrb: hmmm, do you know if this was the same VM from last week that caused things to crawl on n21? |
| 10:52 | <@mikegrb> | that I am not sure |
| 10:52 | <bkirkbri> | that's cool |
| 10:52 | <@mikegrb> | will have to check on |
| 10:52 | <@mikegrb> | going to step out for a smoke but will take another look at n21 when I come back in see how things are going |
| 10:52 | <bkirkbri> | i'm wondering what my best option would be to avoid this... this node runs our support helpdesk |
| 10:52 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe1b:abd3] has quit [Ping timeout: 480 seconds] |
| 10:52 | <bkirkbri> | mikegrb: k |
| 10:53 | <bd_> | arachnid: PTHREAD_MUTEX_INITIALIZER is likely to use futexes, which are very low-overhead |
| 10:54 | -!- | spasmface [~spasm@r220-101-68-180.cpe.unwired.net.au] has quit [Ping timeout: 480 seconds] |
| 10:54 | <bd_> | and by low overhead I mean it won't even enter the kernel unless the lock is contended (it does make use of atomic operations, however, which have a certain degree of overhead) |
| 10:54 | <arachnid> | bd_: Right. so the overhead is in _using_ them, not _creating_ them? |
| 10:54 | <bd_> | arachnid: as long as it's the fast mutex type, anyway |
| 10:54 | <arachnid> | I ask because this FUSE FS could potentially have hundreds of thousands of files, each with a mutex |
| 10:54 | <bd_> | (I think) |
| 10:54 | <arachnid> | It would be, yes |
| 10:55 | <bd_> | If you're really concerned you can always read man 7 futex and implement the protocol there :) |
| 10:55 | <arachnid> | heh |
| 10:55 | <bd_> | The overhead for a raw futex is a whopping four bytes |
| 10:55 | <arachnid> | I'm using glib, which uses posix mutexes under the hood |
| 10:55 | <arachnid> | I'd rather not use them directly. |
| 10:56 | * | bd_ pokes through glibc's sources a bit |
| 10:56 | <arachnid> | It's less a few bytes of space overhead that I'm concerned about - I thought it might require kernel calls, or use some other limited resource, sysv style. |
| 10:57 | <bd_> | well, futexes (again, used as the basic primitive these days) don't even enter the kernel unless contended, and don't consume kernel resources unless they have threads sleeping on them |
| 10:57 | <bd_> | (or rather, with no sleepers their resources are reclaimable) |
| 10:57 | <arachnid> | cool |
| 10:57 | <arachnid> | Any idea if it uses futexes for condition variables too? |
| 10:57 | <arachnid> | (where it == posix) |
| 10:58 | <bd_> | it seems to have some futex stuff in glibc/nptl/pthread_cond_broadcast.c |
| 10:58 | <arachnid> | glibc != glib |
| 10:58 | <arachnid> | Oh, I see, you're looking at how glibc implements posix mutexes |
| 10:58 | <@mikegrb> | bkirkbri: we are looking at a few options to eliminate the issue, but if need be in the meantime, we can always migrate the node to less loaded host |
| 10:59 | <arachnid> | The existence of fast/recursive/error-detecting variants of posix mutexes would certainly tend to indicate that 'fast' probably uses futexes |
| 10:59 | <bd_> | arachnid: yes :) |
| 10:59 | <bd_> | well, they all probably use futexes eventually |
| 10:59 | -!- | Schroeder [1000@kntpin04-nas-02-s76.cinergycom.net] has joined #linode |
| 10:59 | <bkirkbri> | mikegrb: that would be WAY cool, is there a host in Newark that has been less loaded than this one? |
| 11:00 | <arachnid> | Not quite sure if you _could_ implement a condition variable with a futex or not |
| 11:00 | <Peng_> | All of them, probably. |
| 11:00 | <bd_> | arachnid: they do, in actuality :) |
| 11:00 | <arachnid> | But I'll be using far fewer of them than mutexes anyway |
| 11:00 | <arachnid> | Ah, cool |
| 11:00 | <arachnid> | Thanks for the help. :) |
| 11:01 | <bd_> | /* Wait until woken by signal or broadcast. */ |
| 11:01 | <bd_> | lll_futex_wait (&cond->__data.__futex, futex_val, pshared); |
| 11:01 | <bd_> | etc :) |
| 11:03 | <arachnid> | neat |
| 11:03 | <arachnid> | Now I just have to figure out if this really is the best locking scheme ;) |
| 11:03 | <bd_> | remember, atomic decrements aren't free :) |
| 11:03 | <bd_> | relatively cheap, perhaps, but not free :P |
| 11:03 | -!- | bliblok_ [~bjornar@ti500720a080-8908.bb.online.no] has quit [Server closed connection] |
| 11:04 | -!- | bliblok_ [~bjornar@ti500720a080-8908.bb.online.no] has joined #linode |
| 11:04 | <arachnid> | Yeah, but that's the business with _using_ them - actually using one is relatively rare compared to the number I have to create. |
| 11:05 | <bkirkbri> | mikegrb: we can upgrade to a 540 or 720 if that helps to find a host with lower load |
| 11:05 | <@mikegrb> | got a host for you, migration set up and ticket updated |
| 11:06 | <straterra> | 1440 ftw |
| 11:06 | * | bd_ afk for a bit |
| 11:06 | <bkirkbri> | mikegrb: awesome, thanks so much... no IP change correct? |
| 11:07 | -!- | spasmface [~spasm@r220-101-68-180.cpe.unwired.net.au] has joined #linode |
| 11:07 | <@mikegrb> | bkirkbri: correct |
| 11:08 | <@mikegrb> | there is less io contention on the higher plans, that said the level of io contention you are seeing isn't okay with us ;) |
| 11:08 | * | avongauss stops encoding videos again on his Linode. |
| 11:09 | <bkirkbri> | mikegrb: you guys are great! i'm off to start migrating, best of luck with the fix to limit this in the future... |
| 11:15 | -!- | spasmface [~spasm@r220-101-68-180.cpe.unwired.net.au] has quit [Ping timeout: 480 seconds] |
| 11:18 | -!- | bkirkbri [~7a1d4975@webuser.linode.com] has quit [Remote host closed the connection] |
| 11:19 | <sungo> | you know. sometimes the usage graphs scare the crap out of me. |
| 11:19 | <sungo> | right now, my cpu graph looks like I'm using all kinds of cpu. until you realize the scale is 200m - 1000m |
| 11:20 | <Dave> | 200m? |
| 11:21 | <sungo> | as in very very small measuremens of percentages :) |
| 11:21 | <sungo> | milli or micro percentage. I forget which but either way, very very small :) |
| 11:23 | <Dave> | oh, not 200 meters |
| 11:23 | <Dave> | right |
| 11:23 | <Dave> | that makes more sense |
| 11:24 | <HoopyCat> | i recommend setting a cron job to compute a few hundred thousand digits of pi once a day, to fix the scale |
| 11:24 | <DeepInTheFire> | sungo it is milli |
| 11:25 | <Dave> | or just put folding at home on there, keep it at 100% or even 200%! |
| 11:25 | -!- | pbryan [~pbryan@S010600095baae0ff.vf.shawcable.net] has quit [Quit: Leaving.] |
| 11:26 | <DeepInTheFire> | micro would be denoted as u or lowercase upsilon |
| 11:27 | <scorche|sh> | mu! |
| 11:27 | <Dave> | *cough* geeks *cough* |
| 11:30 | <Nivex> | wouldn't that be mu? μ |
| 11:31 | * | tjfontaine points at dave |
| 11:31 | <Nivex> | 300μΩ |
| 11:31 | <Dave> | tjfontaine: you calling me small? |
| 11:31 | <@mikegrb> | yes |
| 11:32 | <HoopyCat> | greek cows say mu |
| 11:32 | <Nivex> | I thought those were cats |
| 11:32 | -!- | sveiss [~sveiss@2001:470:1f09:2a4:21b:63ff:fe1b:abd3] has joined #linode |
| 11:32 | <HoopyCat> | i've got plenty of cats here, and none of them say mu. |
| 11:33 | <Nivex> | they must be mute cats |
| 11:33 | <Nivex> | μ is pronounced 'mew' |
| 11:34 | <HoopyCat> | they're communist, actually; they mostly say "Mao" |
| 11:34 | <tjfontaine> | moo |
| 11:35 | <Nivex> | π |
| 11:42 | <Nivex> | cowpiss == μπ ? |
| 11:42 | <HoopyCat> | heh heh, he said pee. |
| 11:45 | -!- | MrRx7 [~Owner@dpc674448253.direcpc.com] has joined #linode |
| 11:47 | -!- | xitology [~xi@golovko1.donbass.com] has quit [Quit: Ex-Chat] |
| 11:47 | -!- | funkytastic [~9dfc1089@webuser.linode.com] has joined #linode |
| 11:48 | <funkytastic> | hi all |
| 11:49 | -!- | jm [~fake@c-76-113-194-7.hsd1.mn.comcast.net] has quit [] |
| 11:49 | <MrRx7> | hi |
| 11:50 | <funkytastic> | i'm having some problems with my linode. created a new debian 4.0 image and booted from it, now it's not responding to ping |
| 11:50 | <funkytastic> | although the control panel says it's running.; |
| 11:50 | <funkytastic> | any ideas? |
| 11:50 | <MrRx7> | Can you get in via lish? |
| 11:50 | -!- | digx [~rick@131.94.183.248] has quit [Quit: digx] |
| 11:51 | <funkytastic> | lemme try |
| 11:52 | <funkytastic> | shoot, it's rejecting my password. it's my account user/pass right |
| 11:53 | <@mikegrb> | lish password is separate |
| 11:54 | <@mikegrb> | once you get to your linode's console though you will need user/pass for the linode |
| 11:54 | <@mikegrb> | if it's new that's root/pass you entered in distro wizard |
| 11:54 | <funkytastic> | yeah |
| 11:55 | <funkytastic> | ok, i'm in. good call |
| 11:55 | <funkytastic> | it should respond to ping by default, right? |
| 11:56 | <funkytastic> | wtf |
| 11:56 | <bd_> | funkytastic: yes |
| 11:56 | <bd_> | there's no firewall or whatever set up by default |
| 11:56 | -!- | Unhelpful [~9dfc108c@webuser.linode.com] has joined #linode |
| 11:56 | <bd_> | however, your own ISP may be filtering it |
| 11:56 | <bd_> | try pinging google.com from inside your linode |
| 11:56 | -!- | pbryan [~pbryan@209.52.48.162] has joined #linode |
| 11:57 | <funkytastic> | everything was working fine until i wiped just now |
| 11:57 | <funkytastic> | so it's not my isp |
| 11:57 | <funkytastic> | pinging google works fine from the linode |
| 11:58 | <funkytastic> | and i can ping my own home machine |
| 11:59 | <funkytastic> | ok found the problem |
| 11:59 | <funkytastic> | machine's responding on one of my ip addresses, but not the other |
| 11:59 | <Unhelpful> | I think that you need to run "sudo rm -rf /" but I'm not quite sure. |
| 11:59 | <HoopyCat> | Unhelpful, that's unhelpf--- oh, wait. n/m |
| 12:00 | <Unhelpful> | Or maybe do ":(){ : | :& };:" |
| 12:00 | <funkytastic> | Unhelpful: i know where you sleep. |
| 12:00 | <HoopyCat> | funkytastic: only your primary IP is delivered via DHCP; your secondary IP must be statically configured. (while you're at it, you can statically configure your primary too and remove one point of failure) |
| 12:01 | <funkytastic> | neat. makes sense. |
| 12:01 | -!- | rick111_afk [~BigRyan@ryanpartington.com] has quit [Quit: boobies] |
| 12:01 | <Unhelpful> | Also, you need to have donkey installed. Do "apt-get install donkey". |
| 12:01 | <funkytastic> | tips how to set taht up? |
| 12:01 | <funkytastic> | not donkey |
| 12:01 | <funkytastic> | jackass |
| 12:01 | <Unhelpful> | :) |
| 12:02 | <HoopyCat> | i wish more people had obvious nicknames indicating their purpose. <DrunkHomelessGuyWhoWillAskYouForCash298192> Hey man, how's it going? |
| 12:02 | <Unhelpful> | Hey funkytastic, whatcha weaing? |
| 12:02 | <funkytastic> | haha |
| 12:02 | -!- | Schroeder [1000@kntpin04-nas-02-s76.cinergycom.net] has quit [Ping timeout: 480 seconds] |
| 12:02 | <HoopyCat> | funkytastic: /etc/network/interfaces iirc |
| 12:02 | <funkytastic> | sweet |
| 12:02 | -!- | djweezy [djweezy@weezy.us] has joined #linode |
| 12:02 | <Unhelpful> | Like honey. |
| 12:02 | <djweezy> | anyone else having problems updating dns? |
| 12:04 | <djweezy> | :( |
| 12:04 | <Unhelpful> | I love you SpaceHobo |
| 12:05 | <djweezy> | i keep getting an error when i try to edit one of my domains |
| 12:05 | <Battousai> | are you doing it wrong? |
| 12:05 | <Unhelpful> | yo, spacehobo: http://gnupedia.org/ |
| 12:06 | <Unhelpful> | The scary thing is that funkytastic does know where I sleep. |
| 12:06 | <Unhelpful> | (with his mom) |
| 12:07 | <Unhelpful> | Didja click? |
| 12:07 | <djweezy> | http://pastebin.com/m288ce40d |
| 12:07 | -!- | SelfishMan [~TheSelfis@host-72-175-55-117.lvt-mt.client.bresnan.net] has quit [Quit: SelfishMan] |
| 12:07 | <Unhelpful> | OH yeah, it's on funkytastic's linode |
| 12:07 | <Unhelpful> | which is down |
| 12:07 | <Unhelpful> | because he pwned himself |
| 12:07 | <funkytastic> | for the record, i did not |
| 12:08 | <Battousai> | the record is a lie |
| 12:08 | <Unhelpful> | (I actually hax0red him) |
| 12:08 | <Unhelpful> | but don't tell him |
| 12:08 | <funkytastic> | i'm right here |
| 12:08 | <Unhelpful> | Within striking distance you are. |
| 12:09 | <Unhelpful> | Or stroking distance </creepy> |
| 12:09 | <Battousai> | ooh |
| 12:09 | <funkytastic> | i'm arming myself. |
| 12:10 | <funkytastic> | HoopyCat: once i edit the interfaces file, reboot? |
| 12:10 | <Unhelpful> | Lunchtime. |
| 12:10 | <Unhelpful> | Rice. |
| 12:10 | <Unhelpful> | Let's roll out, funkytastic |
| 12:10 | <funkytastic> | hold yer freakin horses |
| 12:11 | -!- | exor674 is now known as exor|zzz |
| 12:12 | <funkytastic> | ok lunchtime. back later |
| 12:12 | <funkytastic> | thanks for the help |
| 12:18 | -!- | jm [~fake@c-76-113-194-7.hsd1.mn.comcast.net] has joined #linode |
| 12:23 | -!- | linville [~linville@205.233.54.157] has joined #linode |
| 12:24 | <cruxeternus> | So, what was the official word about the network outage/issues last night ~5PM ET? |
| 12:24 | <Battousai> | ddos i think |
| 12:25 | <cruxeternus> | That was the theory at the time... just wondering if there was any official update on it. |
| 12:26 | -!- | SelfishMan [~TheSelfis@69.51.75.42] has joined #linode |
| 12:27 | -!- | Netsplit resistance.oftc.net <-> charon.oftc.net quits: row, Karrde, binel, cramer_as8758, Karnaugh, Ascii_, Clorith, SNy, maqr, paulcager, (+4 more, use /NETSPLIT to show all of them) |
| 12:27 | -!- | Netsplit larich.oftc.net <-> osmotic.oftc.net quits: jkwood, Pryon, Toba, arachnid, gpd, neoark, jcr, meff, mattbnz, [prag]greg, (+121 more, use /NETSPLIT to show all of them) |
| 12:27 | -!- | Netsplit over, joins: SelfishMan |
| 12:27 | -!- | Netsplit over, joins: jm |
| 12:27 | -!- | Netsplit over, joins: pbryan |
| 12:27 | -!- | Netsplit over, joins: Unhelpful |
| 12:27 | -!- | Netsplit over, joins: funkytastic |
| 12:28 | -!- | Netsplit over, joins: sveiss |
| 12:28 | -!- | Netsplit over, joins: bliblok_ |
| 12:28 | -!- | Netsplit over, joins: lakin |
| 12:28 | -!- | Netsplit over, joins: TimothyA |
| 12:28 | -!- | Netsplit over, joins: jcr |
| 12:28 | -!- | Netsplit over, joins: JDLSpeedy |
| 12:28 | -!- | Netsplit over, joins: MrGeneral |
| 12:28 | -!- | Netsplit over, joins: internat1 |
| 12:28 | -!- | Netsplit over, joins: jimcooncat |
| 12:28 | -!- | Netsplit over, joins: tsp |
| 12:28 | -!- | Netsplit over, joins: besonen_mobile |
| 12:28 | -!- | Netsplit over, joins: TehDan |
| 12:28 | -!- | Netsplit over, joins: metaperl |
| 12:28 | -!- | Netsplit over, joins: arooni-mobile |
| 12:28 | -!- | Netsplit over, joins: Noia |
| 12:28 | -!- | Netsplit over, joins: Kassah |
| 12:28 | -!- | Netsplit over, joins: cdlu |
| 12:28 | -!- | Netsplit over, joins: npmr |
| 12:28 | -!- | Netsplit over, joins: dvgrhl |
| 12:28 | -!- | Netsplit over, joins: Hobbsee |
| 12:28 | -!- | Netsplit over, joins: jkwood |
| 12:28 | -!- | Netsplit over, joins: Dreamr_3 |
| 12:28 | -!- | Netsplit over, joins: [prag]greg |
| 12:28 | -!- | Netsplit over, joins: charlie |
| 12:28 | -!- | Netsplit over, joins: gpd |
| 12:28 | -!- | Netsplit over, joins: zeroday |
| 12:28 | -!- | Netsplit over, joins: Zotnix |
| 12:28 | -!- | Netsplit over, joins: HoopyCat |
| 12:28 | -!- | Netsplit over, joins: emag |
| 12:28 | -!- | Netsplit over, joins: ido |
| 12:28 | -!- | Netsplit over, joins: tierra |
| 12:28 | -!- | Netsplit over, joins: JasonF |
| 12:28 | -!- | Netsplit over, joins: chesty |
| 12:28 | -!- | Netsplit over, joins: BubbleWrap |
| 12:28 | -!- | Netsplit over, joins: W|GGL|T |
| 12:28 | -!- | Netsplit over, joins: meff |
| 12:28 | -!- | Netsplit over, joins: LadyNikon |
| 12:28 | -!- | Netsplit over, joins: fo0bar |
| 12:28 | -!- | Netsplit over, joins: Peng__ |
| 12:28 | -!- | Netsplit over, joins: Peng |
| 12:28 | -!- | Netsplit over, joins: lucca |
| 12:28 | -!- | Netsplit over, joins: alnr |
| 12:28 | -!- | Netsplit over, joins: pointer |
| 12:28 | -!- | Netsplit over, joins: det |
| 12:28 | -!- | Netsplit over, joins: Nigel |
| 12:28 | -!- | Netsplit over, joins: mattt |
| 12:28 | -!- | Netsplit over, joins: Syn- |
| 12:28 | -!- | Netsplit over, joins: Bdragon |
| 12:28 | -!- | Netsplit over, joins: l0uis |
| 12:28 | -!- | Netsplit over, joins: sungo |
| 12:28 | -!- | Netsplit over, joins: schmichael |
| 12:28 | -!- | Netsplit over, joins: Pryon |
| 12:28 | -!- | Netsplit over, joins: Redgore |
| 12:28 | -!- | Netsplit over, joins: mattbnz |
| 12:28 | -!- | Netsplit over, joins: warewolf |
| 12:28 | -!- | Netsplit over, joins: linbot |
| 12:28 | -!- | Netsplit over, joins: taupehat |
| 12:28 | -!- | Netsplit over, joins: blw |
| 12:28 | -!- | Netsplit over, joins: tjfontaine |
| 12:28 | -!- | Netsplit over, joins: hachi |
| 12:28 | -!- | Netsplit over, joins: CaptObviousman |
| 12:28 | -!- | Netsplit over, joins: bronson |
| 12:28 | -!- | Netsplit over, joins: GMFlash |
| 12:28 | -!- | Netsplit over, joins: PAtrik |
| 12:28 | -!- | Netsplit over, joins: praetorian |
| 12:28 | -!- | Netsplit over, joins: tozz |
| 12:28 | -!- | Netsplit over, joins: guinea-pig |
| 12:28 | -!- | Netsplit over, joins: BrianHV |
| 12:28 | -!- | Netsplit over, joins: mariorz |
| 12:28 | -!- | Netsplit over, joins: wgrant |
| 12:28 | -!- | Netsplit over, joins: privet |
| 12:28 | -!- | Netsplit over, joins: SeishinMoon |
| 12:28 | -!- | Netsplit over, joins: neh |
| 12:28 | -!- | Netsplit over, joins: avongauss |
| 12:28 | -!- | Netsplit over, joins: petec_ |
| 12:28 | -!- | Netsplit over, joins: darkbeholder |
| 12:28 | -!- | Netsplit over, joins: ajmitch |
| 12:28 | -!- | Netsplit over, joins: Nat_RH |
| 12:28 | -!- | Netsplit over, joins: Peng_ |
| 12:28 | -!- | Netsplit over, joins: DennisTT |
| 12:28 | -!- | Netsplit over, joins: brc |
| 12:28 | -!- | Netsplit over, joins: jadoba |
| 12:28 | -!- | Netsplit over, joins: neoark |
| 12:28 | -!- | Netsplit over, joins: Aero187 |
| 12:28 | -!- | Netsplit over, joins: el_syn |
| 12:28 | -!- | Netsplit over, joins: skule |
| 12:28 | -!- | Netsplit over, joins: path- |
| 12:28 | -!- | Netsplit over, joins: FloodServ |
| 12:28 | -!- | ServerMode/#linode [+oo linbot jadoba] by resistance.oftc.net |
| 12:28 | -!- | Netsplit over, joins: elky |
| 12:28 | -!- | Netsplit over, joins: opello |
| 12:28 | -!- | Netsplit over, joins: pygmalion |
| 12:28 | -!- | Netsplit over, joins: Nivex |
| 12:28 | -!- | Netsplit over, joins: purrdeta |
| 12:28 | -!- | Netsplit over, joins: ElectricElf |
| 12:28 | -!- | Netsplit over, joins: Zachg |
| 12:28 | -!- | Netsplit over, joins: StevenK |
| 12:28 | -!- | Netsplit over, joins: straterra |
| 12:28 | -!- | Netsplit over, joins: cruxeternus |
| 12:28 | -!- | Netsplit over, joins: lanstin |
| 12:28 | -!- | Netsplit over, joins: asedeno |
| 12:28 | -!- | Netsplit over, joins: jvaughan |
| 12:28 | -!- | Netsplit over, joins: scorche |
| 12:28 | -!- | Netsplit over, joins: rmayorga |
| 12:28 | -!- | Netsplit over, joins: Solver |
| 12:28 | -!- | Netsplit over, joins: shakr |
| 12:28 | -!- | Netsplit over, joins: Battousai |
| 12:28 | -!- | Netsplit over, joins: tonyyarusso |
| 12:28 | -!- | Netsplit over, joins: syntaxman |
| 12:28 | -!- | Netsplit over, joins: pleia2 |
| 12:28 | -!- | Netsplit over, joins: bd_ |
| 12:28 | -!- | Netsplit over, joins: jackc |
| 12:28 | -!- | Netsplit over, joins: caker |
| 12:28 | -!- | Netsplit over, joins: tres |
|
