Morrissey-solo network issues

[davidt]

Morrissey-solo is experiencing networking issues which results in intermittent downtime of around 30 minutes at a time. The problems began around Jan. 18.

This is likely hardware related so may take some time to be resolved. Thanks for your patience in the meantime. Updates will be posted in this thread.

[Jan. 23]
New server motherboard has been ordered and is on the way.

[Jan. 24]
Planned downtime for hardware replacement: Thursday (Jan. 24) 8pm - Friday 12pm (Pacific Time)

[Jan. 25]
Hardware has been replaced but networking issues continue. ISP is now investigating as it is fairly certain not an issue on the server side.

No cause for the outages has been found after ISP investigation. They went ahead and gave me a new IP range for the server which is now in use. It may take a day or two for the DNS changes to propagate to all users.

[Not Right in the Head]

Thanks for the update, David. It doesn't bode well for my SoLow Downtime Drinking Game, though, in which I take a double shot every time I refresh and find that the site is down again. :eek:

:D

[Jose]

Thanks for keeping us informed, David. It's much appreciated!

[troubleluvsme]

:
Thanks for the update, David. It doesn't bode well for my SoLow Downtime Drinking Game, though, in which I take a double shot every time I refresh and find that the site is down again. :eek:

:D


That's funny NRitH! I really wish that I had thought of it first :D :clap:

[Missing Link]

Thanks for the update David. Your efforts are appreciated. To some people this is a website, to others it's a life support system!.

Good luck with the problems. Looking to the future when all will be well ;-)

[Buzzetta]

Thanks for the update, David. It doesn't bode well for my SoLow Downtime Drinking Game, though, in which I take a double shot every time I refresh and find that the site is down again. :eek:

:D

You were playing too?

[William4]

ohh so thats whats going on..

[Corrissey]

Thanks, davidt! You know, I'm actually being productive...around 30 minutes at a time :D

[troubleluvsme]

Thanks, davidt! You know, I'm actually being productive...around 30 minutes at a time :D

"They" say that mini breaks do increase productivity.....
This weekend, I rested each time solo was down, and I finally reached 300 posts!!!!!!

[prue4ever]

"They" say that mini breaks do increase productivity.....
This weekend, I rested each time solo was down, and I finally reached 300 posts!!!!!!

i actually did some of my homework during some mini breaks! :eek:

about 20 posts away from me!

[troubleluvsme]

i actually did some of my homework during some mini breaks! :eek:

about 20 posts away from me!


I fed my kids while SoLow was down!!
(REAL food FNIOP---NOT POPCORN!!!)

[Corrissey]

I fed my kids while SoLow was down!!
(REAL food FNIOP---NOT POPCORN!!!)

Where are they now? Back in the closet?! :p

Happy 350. You're gaining on me!!

[prue4ever]

I fed my kids while SoLow was down!!
(REAL food FNIOP---NOT POPCORN!!!)

wow, i must've not been your kid over the past few days, cos i've been eating popcorn alot! :eek:



You're gaining on me!!

almost there :p

[WinkWink]

oh just pull the plug so we all can get on with our lives.

[neonmad]

oh just pull the plug so we all can get on with our lives.

:eek::tears:

If I could do that, I wouldn't be here in the first place. :rolleyes:

I was afraid the site had been shut down! And what with my cd player bein' on the fritz, I am glad to see that I'm not actually "solo".

[troubleluvsme]

[QUOTE=prue4ever;755433]wow, i must've not been your kid over the past few days, cos i've been eating popcorn alot! :eek:


That's what I'm saying Pure.....I need you to babysit my kids...
popcorn and Morrissey. You'd all get along great! Hey, why don't you watch them Friday so that I can go to Smiths Night with Corrissey? :guitar:
(just kidding)

[prue4ever]

[QUOTE=prue4ever;755433]wow, i must've not been your kid over the past few days, cos i've been eating popcorn alot! :eek:


That's what I'm saying Pure.....I need you to babysit my kids...
popcorn and Morrissey. You'd all get along great! Hey, why don't you watch them Friday so that I can go to Smiths Night with Corrissey? :guitar:
(just kidding)

well if you really want me too. lol :p:D:guitar:
i actually would, i have no life. lol (i think morrissey is my only life, and i cant drive yet, lol)
just have a time when you have another babysitter, so i can come to! lol

[troubleluvsme]

[QUOTE=troubleluvsme;755629]

well if you really want me too. lol :p:D:guitar:
i actually would, i have no life. lol (i think morrissey is my only life, and i cant drive yet, lol)
just have a time when you have another babysitter, so i can come to! lol


Well, since you aren't old enough to drive yet, you probably aren't going to be able to get into Smiths Night at a bar :tears:
Perhaps, we could organize a "Cosmic Bowling" Moz/Smiths night!!

[Not Right in the Head]

Well, since you aren't old enough to drive yet, you probably aren't going to be able to get into Smiths Night at a bar :tears:
Perhaps, we could organize a "Cosmic Bowling" Moz/Smiths night!!

Goddamn, that'd be awesome. Sadly both bowling alleys around the corner from me are shut down. :tears:

[prue4ever]

[QUOTE=prue4ever;755647]


Well, since you aren't old enough to drive yet, you probably aren't going to be able to get into Smiths Night at a bar :tears:
Perhaps, we could organize a "Cosmic Bowling" Moz/Smiths night!!

lol, true, i could sneak in though! lol

a few weekends ago, i went cosmic ice skating in carmel. it was fun.

[troubleluvsme]

[QUOTE=troubleluvsme;756073]

lol, true, i could sneak in though! lol

a few weekends ago, i went cosmic ice skating in carmel. it was fun.

Hey, I've been there!!! It's lots of fun!
It's ten minutes from my house!! I may have there that night!
(Did you see anyone with long dark brown hair wearing a Morrissey shirt?:D)

[Not Right in the Head]

Alright, I'm officially checking out of here until some kind soul informs me that the site is back up once and for all.

Please contact me at nrith@tibbetts.net when things are back to normal.

Now let's just see how long I can really stay away.

[vivabob]

it seems to be ok today ive had no issues

[vicarinatutugal]

it seems to be ok today ive had no issues

Its been up and down for me a lot today. Just the same as it was, sort of down for about half hour or so at a time.

[nowherefast944]

Its been up and down for me a lot today. Just the same as it was, sort of down for about half hour or so at a time.

it only seems to be down for me when i'm posting on the forums. then for a half hour or an hour or so after. :(

[PregnantForTheLastTime]

David, is there any update? Want us to take up a collection to buy new hardware?

[joey]

If I have to hop on a plane to come fix a server, I'll do it! I can ship out 100 NIC's by tomorrow if needed to :)

[Kewpie]

David, is there any update? Want us to take up a collection to buy new hardware?


Every bit of help would be welcome :)

http://www.morrissey-solo.com/faq.shtml

[Corrissey]

oh just pull the plug so we all can get on with our lives.

LOL ~ so true :)


[QUOTE=prue4ever;755647]

Perhaps, we could organize a "Cosmic Bowling" Moz/Smiths night!!

I love Cosmic Bowling!! And set to Moz music?! Oscillate down the lanes!
Perhaps I can work on this with DJ Tony & a local bowling alley :cool:


Alright, I'm officially checking out of here until some kind soul informs me that the site is back up once and for all.


Liar ;)

[davidt]

New motherboard is on the way. I'm pretty sure the onboard network card is flaky, let's hope this fixes it.

No need for any collection, the ads pay for stuff like this. But thank you to those who did contribute!


David, is there any update? Want us to take up a collection to buy new hardware?

[davidt]

Planned downtime for the motherboard replacement: Thursday 8pm - Friday 12pm (Pacific Time)

[Kewpie]

Planned downtime for the motherboard replacement: Thursday 8pm - Friday 12pm (Pacific Time)


UK time is Friday 4am - 8am.

[prue4ever]

[QUOTE=prue4ever;756129]

Hey, I've been there!!! It's lots of fun!
It's ten minutes from my house!! I may have there that night!
(Did you see anyone with long dark brown hair wearing a Morrissey shirt?:D)

lol, i dont think i did, but there were a lot of people there, so maybe! lol (Did you see anyone with a black short sleeved polo shirt there? :D)

[davidt]

UK time is Friday 4am - 8am.

sorry, it will be a bit longer - UK time Friday 4am to 8pm. I hope to get the server back up Friday at noon PT (not midnight).

[Hellie]

:eek::tears:

If I could do that, I wouldn't be here in the first place. :rolleyes:

I was afraid the site had been shut down! And what with my cd player bein' on the fritz, I am glad to see that I'm not actually "solo".


There's always You Tube.I can't go a day without it.:)

[Not Right in the Head]

[Jan. 25]
Hardware has been replaced. All looks good so far, we'll see if it holds up.

Dare I? Okay, I will.

[vicarinatutugal]

*fingers crossed*

good luck everyone. :p

[Not Right in the Head]

*fingers crossed*

good luck everyone. :p

Seems a bit slow, doesn't it?

[vicarinatutugal]

Seems a bit slow, doesn't it?

yes very, I have been having some internet issues for a wee while though, so I was not sure if it was that or solo.

[Not Right in the Head]

yes very, I have been having some internet issues for a wee while though, so I was not sure if it was that or solo.

I think it's SoLow. Images, in particular, seem a bit slow to come up. Not always, just sometimes.

[EPbabe]

I think it's SoLow. Images, in particular, seem a bit slow to come up. Not always, just sometimes.

I haven't been able to open my yahoo.com mailbox today.:(

[vicarinatutugal]

I think it's SoLow. Images, in particular, seem a bit slow to come up. Not always, just sometimes.

anyone else think its rude to call it soLow when we are here, I mean we love the site right? :p

[Not Right in the Head]

I haven't been able to open my yahoo.com mailbox today.:(

Hmm, that may not be totally unrelated, actually.

[EPbabe]

Hmm, that may not be totally unrelated, actually.

I thought it wasn't. I mean both come from a US server, don't they? And my youtube account looked weird too...:eek:

[Corrissey]

davidt, solo is running like a dream :) yeay & thanks :guitar:

[joey]

Running absolutely smoothly. Got that dedicated DS3 to it now, right :)

[davidt]

Running smooth for now but when I plugged it in the network wasn't working, then suddenly started working 20 minutes later. Not a good sign! Joey, the ISP may need your help. ;)

[Not Right in the Head]

Running smooth for now but when I plugged it in the network wasn't working, then suddenly started working 20 minutes later. Not a good sign! Joey, the ISP may need your help. ;)

That's a really bad sign. It's not just a flaky cable?

[davidt]

That's a really bad sign. It's not just a flaky cable?

We have tried different cables and moving to a different switch, but we'll have to look at all of that more closely if the problems continue.

[joey]

Can you get into the switch? Check for duplex mismatches if you can get into it. That can cause flaky issues sometimes.

[Not Right in the Head]

We have tried different cables and moving to a different switch, but we'll have to look at all of that more closely if the problems continue.

Have you looked into any offsite hosting options if the problems continue? Even a static site hosted elsewhere is better than a dead dynamic one here. I needed to look up some details about the recent setlists after I downloaded the recordings, and the site was down...maybe that's a problem only for me. I do appreciate all you're doing to get everything back up.

[davidt]

Can you get into the switch? Check for duplex mismatches if you can get into it. That can cause flaky issues sometimes.

I'll suggest that to the ISP but they swear there have been no recent changes. there are 15 or so other servers on the same switch and they are all running fine, so it's a mystery. We have changed ports and even moved to another switch. So I think it may be something with their provisioning causing an IP conflict.

[Not Right in the Head]

there are 15 or so other servers on the same switch and they are all running fine, so it's a mystery.

Do you have physical access to the server? Because if so, I can think of 15 easy things to do to solve the problem. ;)

[davidt]

Have you looked into any offsite hosting options if the problems continue? Even a static site hosted elsewhere is better than a dead dynamic one here. I needed to look up some details about the recent setlists after I downloaded the recordings, and the site was down...maybe that's a problem only for me. I do appreciate all you're doing to get everything back up.

I've been looking at the new virtual server service Amazon started offering (Elastic Computing Cloud) but that is going to take a lot of work to move to. A static site is also more work. Since the site has been stable for the last several years, I don't think there is need to panic... yet.

[joey]

Just admit it, the frink thread crashed it :)

I just looked at Amazon EC2, looks pretty cool, but still in beta. Might be worth a test site.

[davidt]

Just admit it, the frink thread crashed it :)

I just looked at Amazon EC2, looks pretty cool, but still in beta. Might be worth a test site.

That's true, the problems did start around the time the limit of frink images was raised from 4 to 8 per post. :horny:

EC2 is still very early but it seems like the future. I like the idea of not buying hardware and if something goes wrong, just turning on another instance.

[goinghome]

"Stop!
I said it's happening again!
We're all wasting away!
We're all wasting away!"
Empire - Kasabian

Good luck with ameliorative efforts!

[davidt]

Yes, it is still happening. I'm working with the ISP on it, they have just swapped cables with my other box which has been fine (images server). We'll get to the bottom of this soon!

[vivabob]

Yes, it is still happening. I'm working with the ISP on it, they have just swapped cables with my other box which has been fine (images server). We'll get to the bottom of this soon!

thanks for all your effort ... im sure i speak on behalf of everyone

[JoyDiv007]

I'm still having problems but I understand about all the problems. Don't worry about it.

[davidt]

We tried everything and could not figure out the issue. As a last resort the ISP gave me a new IP range to move to, which I just switched to. It will take a day or two for the DNS changes to propagate to everyone.

[davidt]

The outages may be related to recent hacker DDOS attacks on Scientology sites. Investigation continues.

[withmyheadonthebar]

The outages may be related to recent hacker DDOS attacks on Scientology sites. Investigation continues.

:confused::confused::confused::confused:

[davidt]

Sorry, some more background, there is currently a massive hacker attack on scientology sites:

Hackers Hit Scientology With Online Attack (http://news.yahoo.com/s/pcworld/20080126/tc_pcworld/141839)

Church of Scientology DDoS Statistics (http://asert.arbornetworks.com/2008/01/church-of-scientology-ddos-statistics)

Maximum duration of a single attack: 1.8 hours, which is on the long end of common, but the average attack lasted just under half an hour

--
The hackers may have hacked the Morrissey-solo server and have some script running attacking the scientology sites and causing the outages. The average attack under half an hour is consistent with the duration of the outages. I'm investigating the server but haven't found anything yet.

[joey]

So can we blame Tom Cruise for this?

Seriously though, thank you so much for all of your help in keeping up this site.

[Not Right in the Head]

Sorry, some more background, there is currently a massive hacker attack on scientology sites:

Hackers Hit Scientology With Online Attack (http://news.yahoo.com/s/pcworld/20080126/tc_pcworld/141839)

Church of Scientology DDoS Statistics (http://asert.arbornetworks.com/2008/01/church-of-scientology-ddos-statistics)

Maximum duration of a single attack: 1.8 hours, which is on the long end of common, but the average attack lasted just under half an hour

--
The hackers may have hacked the Morrissey-solo server and have some script running attacking the scientology sites and causing the outages. The average attack under half an hour is consistent with the duration of the outages. I'm investigating the server but haven't found anything yet.

Hmmm, I told someone offline early on that it looked like a DDOS, but it seems pretty unlikely that hackers put a script on the SoLow server in order to attack Scientology sites. What if the Scientologists are hitting SoLow with a DDOS for the Tom Cruise thread last week? Even if that's just a wild conjecture, it sounds more intriguing than something like a flaky NIC or misallocated IP addresses. :D


So can we blame Tom Cruise for this?

Sounds good to me.

[laughing_anne]

Hmmm, I told someone offline early on that it looked like a DDOS, but it seems pretty unlikely that hackers put a script on the SoLow server in order to attack Scientology sites. What if the Scientologists are hitting SoLow with a DDOS for the Tom Cruise thread last week? Even if that's just a wild conjecture, it sounds more intriguing than something like a flaky NIC or misallocated IP addresses. :D



Sounds good to me.

I just love a good conspiracy theory! :D

[vicarinatutugal]

Hmmm, I told someone offline early on that it looked like a DDOS, but it seems pretty unlikely that hackers put a script on the SoLow server in order to attack Scientology sites. What if the Scientologists are hitting SoLow with a DDOS for the Tom Cruise thread last week? Even if that's just a wild conjecture, it sounds more intriguing than something like a flaky NIC or misallocated IP addresses. :D





Oh christ! it's my fault then, a total frinker and Tom cruise/scientology distruster... ooops.

[davidt]

Hmmm, I told someone offline early on that it looked like a DDOS, but it seems pretty unlikely that hackers put a script on the SoLow server in order to attack Scientology sites. What if the Scientologists are hitting SoLow with a DDOS for the Tom Cruise thread last week? Even if that's just a wild conjecture, it sounds more intriguing than something like a flaky NIC or misallocated IP addresses. :D


Both scenarios are possible - usually the ISP catches any DDOS network activity and either blocks the traffic coming in or lets me know the server is compromised because they see the DDOS traffic going out but in this case said they did not see anything at all. It's most likely on the server end so after the investigation I will likely have to wipe the system.

The hackers have been evolving and these attacks are more difficult to detect nowadays.

[Bassist-In-A-Tutu]

So, you have new MB, new NIC (You did replace the NIC?), new cable, new switch and new IP Address - what a great mistery! :confused:

Could it be an issue on your ISP's DNS? Any other web sites on this server? If so, how are they? And none of the other servers on your switch have this problem - are they on the same IP subnet as you?

I love a good mistery! (Sorry to get all involved, but I have been working in IT for 20 years and am now an IT Manager, so enjoy joining in with this sort of thing - tell me to get lost if you like...) lol

:)

[davidt]

So, you have new MB, new NIC (You did replace the NIC?), new cable, new switch and new IP Address - what a great mistery! :confused:

Could it be an issue on your ISP's DNS? Any other web sites on this server? If so, how are they? And none of the other servers on your switch have this problem - are they on the same IP subnet as you?

I love a good mistery! (Sorry to get all involved, but I have been working in IT for 20 years and am now an IT Manager, so enjoy joining in with this sort of thing - tell me to get lost if you like...) lol

:)

I believe it was definitely a DOS of some sort (either inbound or outbound, haven't determined yet). Changing IP stopped the problems so the DOS was targeting that IP and not anything to do with DNS. That is correct, there are several other servers and websites on the subnet but no other reported problems.

I replaced the motherboard because the network interface is integrated into the board. In retrospect I could have fooled around a bit more with adding network cards but I just wanted to fix the problem as soon as possible on the live system and not have to keep driving back and forth to the ISP. It would have saved a lot of trouble if the ISP had caught the DDOS network activity. Hackers are getting advanced with their attacks, but I am assured that the ISP is doing their best.

So now I have an extra motherboard, which I think I will use to build a new system to replace the other server which is about 8 years old.

[vicarinatutugal]

Anyone else not getting solo emails? subcribed thread ones, so on?

[vicarinatutugal]

I am. No problems there for me. Sorry.

hmm thanks for the feedback, I still am not. Weird.

[davidt]

hmm thanks for the feedback, I still am not. Weird.

I'm seeing returned messages from your ISP because of the IP change. I'll forward you the details.

[vicarinatutugal]

I'm seeing returned messages from your ISP because of the IP change. I'll forward you the details.

ok thanks david :)

[gperkins151]

Anyone else not getting solo emails? subcribed thread ones, so on?

I am not getting them. :(

BTW, is is possible to get emails for Private Messages?

[Not Right in the Head]

I am not getting them. :(

BTW, is is possible to get emails for Private Messages?

Yes. User CP > Edit Options > Messaging & Notification > check Receive Email Notification of New Private Messages

I don't know why it's not the default mode.

[gperkins151]

Yes. User CP > Edit Options > Messaging & Notification > check Receive Email Notification of New Private Messages

I don't know why it's not the default mode.

Thanks!

[gperkins151]

I'm seeing returned messages from your ISP because of the IP change. I'll forward you the details.

Could you please look into my situation?

I am not getting email notification either.

[EPbabe]

Anyone else not getting solo emails? subcribed thread ones, so on?

I am getting them but with hours of delay and treated as spams.

[davidt]

IP change / mail server configuration issues will be fixed by the ISP, hopefully within the next few days.

[EPbabe]

Not complaining, David, I'm thickled pink that the site works again! :D

[davidt]

Not complaining, David, I'm thickled pink that the site works again! :D

No problem, just need to fix some issues related to the IP change and emails should be back to how it was.

[gperkins151]

IP change / mail server configuration issues will be fixed by the ISP, hopefully within the next few days.

GREAT!

Thanks. :)

[youarethequarry]

Sorry, some more background, there is currently a massive hacker attack on scientology sites:

Hackers Hit Scientology With Online Attack (http://news.yahoo.com/s/pcworld/20080126/tc_pcworld/141839)

Church of Scientology DDoS Statistics (http://asert.arbornetworks.com/2008/01/church-of-scientology-ddos-statistics)

Maximum duration of a single attack: 1.8 hours, which is on the long end of common, but the average attack lasted just under half an hour

--
The hackers may have hacked the Morrissey-solo server and have some script running attacking the scientology sites and causing the outages. The average attack under half an hour is consistent with the duration of the outages. I'm investigating the server but haven't found anything yet.

DavidT, you're a very successful man. Have you ever thought about what might be holding you back from achieving even greater success and wealth? Why is your web site acting up just as Morrissey embarks upon a European tour?

There is a single source of all your problems, stress, unhappiness and self-doubt. It’s called the reactive mind - the hidden part of your mind that stores all painful experiences and then uses them against you. Read Dianetics and discover the real reason for unexplained pains, negative emotions and unhappy relationships in your life. Exactly what is destroying your belief in yourself and how Dianetics helps you get rid of it so you become more you; and the precise technology that blows away the barriers in your life - forever.

[vicarinatutugal]

No problem, just need to fix some issues related to the IP change and emails should be back to how it was.

Thanks David, email working and all 600 odd of them have arrived! :eek:

[gperkins151]

Thanks David, email working and all 600 odd of them have arrived! :eek:

I got one, so I think that mine might be still not working. :(

[EPbabe]

I still have email issues: Solo notifications arrive with even several days of delay and they're always treated as spams by yahoo. :(

[Kewpie]

I still have email issues: Solo notifications arrive with even several days of delay and they're always treated as spams by yahoo. :(


Hmmm...have you asked yahoo for advice?

[EPbabe]

Hmmm...have you asked yahoo for advice?

No, I haven't , because I have no problems with any other emails. Only the Solo ones.

[Grim O'Grady]

they are a cunt imho.

love

Grim

[Dave]

No, I haven't , because I have no problems with any other emails. Only the Solo ones.

I haven't used yahoo in a while, but they probably have a "safe sender" list, or an option to add solo to your address book, which usually lets the mail through.

Unless solo is trying to sell v1agr@ or something.

[davidt]

I still have email issues: Solo notifications arrive with even several days of delay and they're always treated as spams by yahoo. :(

I continue to see 'Message temporarily deferred' messages for your emails, but no one else is getting that message. I'll look into it some more.

[EPbabe]

I haven't used yahoo in a while, but they probably have a "safe sender" list, or an option to add solo to your address book, which usually lets the mail through.

Unless solo is trying to sell v1agr@ or something.

Lol, you never know! :)


I continue to see 'Message temporarily deferred' messages for your emails, but no one else is getting that message. I'll look into it some more.

Thanks a lot David. :)

[Patrick McGoohan]

DavidT, you're a very successful man. Have you ever thought about what might be holding you back from achieving even greater success and wealth? Why is your web site acting up just as Morrissey embarks upon a European tour?

There is a single source of all your problems, stress, unhappiness and self-doubt. It’s called the reactive mind - the hidden part of your mind that stores all painful experiences and then uses them against you. Read Dianetics and discover the real reason for unexplained pains, negative emotions and unhappy relationships in your life. Exactly what is destroying your belief in yourself and how Dianetics helps you get rid of it so you become more you; and the precise technology that blows away the barriers in your life - forever.

Holy crap! Did we get a Genuine Scientologist on here?! Amazing!
Can someone get Tom Cruise on the line???

[9 x Fined]

Hi Dave. No complaints. Thanks for not banning me. Actually, can I be a mod.