Editors • About the Site • Comes vs. Microsoft • Using This Web Site • Site Archives • Credibility Index • OOXML • OpenDocument • Patents • Novell • News Digest • Site News • RSS

02.03.09

Call for Advice and Help

Posted in Boycott Novell, Site News at 11:43 am by Shane Coyle

Hey folks, I need to lean a bit on our readership here and call for any and all suggestions for helping the site become more slashdot-proof.

We’re at the point of perhaps needing dedicated hosting, so maybe we’ll need suggestions for that as well, since my host company is getting tired of melted machines.

Too many http connections
————–
One of your resold accounts (boycottnovell.com) was causing a considerable load on the server which your reseller account is hosted on. When I investigated this issue further, I discovered that this account is averaging more than 180,000 hits per day, with spikes as high as 700,000 hits per day. This is far too much traffic for a shared hosting environment.

I still believe that we can do this with proper caching and design, and I’m sure that it’s a problem that’s been solved before. Any suggestions?

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • StumbleUpon
  • Reddit
  • co.mments
  • DZone
  • email
  • Google Bookmarks
  • LinkedIn
  • NewsVine
  • Print
  • Propeller
  • Slashdot
  • Technorati
  • TwitThis
  • Webnews
  • YahooMyWeb

If you liked this post, consider subscribing to the RSS feed or join us now at the IRC channel. To use your own IRC client, join channel #boycottnovell in FreeNode.

Pages that cross-reference this one

4 Comments

  1. Roy Schestowitz said,

    February 3, 2009 at 12:40 pm

    Gravatar

    Hi, Shane.

    Thanks for putting time into this. There was a discussion about it over the weekend (IRC) and minutes ago it came up again. Maybe the log below will help because it discusses options. Let me know what you think.

    From the past ~30 minutes:

    Feb 03 17:03:03 <The_Mad_Hatter>Try PRQ for hosting
    The_Mad_Hatter http://en.wikipedia.org/wiki/PRQ Feb 03 17:03
    The_Mad_Hatter They host wikileaks among others. Feb 03 17:03
    The_Mad_Hatter And are not likely to bow down to Microsoft. Feb 03 17:03
    schestowitz The_Mad_Hatter: thanks. Feb 03 17:07
    schestowitz I considered Squid Feb 03 17:07
    schestowitz I need to get it installed once someone who advises me gets back to me. He was here in IRC the other day Feb 03 17:07
    schestowitz You advised this to me before, right? Feb 03 17:08
    schestowitz I think it’s you who did. Feb 03 17:08
    The_Mad_Hatter Um, not directly. I did mention that I knew a place that was good. Feb 03 17:09
    schestowitz Pirate Bay is bad IP (Address) neighbourhood, no? Feb 03 17:10
    The_Mad_Hatter Not as far as I can tell. Feb 03 17:10
    schestowitz Another person suggested ibiblio Feb 03 17:10
    schestowitz They host Groklaw, which has a dedicated AMD box Feb 03 17:11
    The_Mad_Hatter Ibibilio would be good to. You need something that can Feb 03 17:11
    The_Mad_Hatter A – Handle the load Feb 03 17:11
    The_Mad_Hatter B – Won’t be pushed around Feb 03 17:11
    ushimitsudoki what up fellas Feb 03 17:11
    schestowitz Hey, ushimitsudoki Feb 03 17:11
    ushimitsudoki I got laid off today Feb 03 17:11
    schestowitz We need advice: http://boycottnovell.com/2009/02/03/call-fo... Feb 03 17:11
    The_Mad_Hatter PRQ and Ibiblio are both good that way. Feb 03 17:11
    The_Mad_Hatter Morning. Feb 03 17:11
    schestowitz ushimitsudoki: many people did. Feb 03 17:11
    schestowitz It’s not you, it’s the economy Feb 03 17:12
    The_Mad_Hatter Ouch. Sorry to hear that. Feb 03 17:12
    The_Mad_Hatter Of course then you got guys like me who can’t work. Feb 03 17:12
    schestowitz The_Mad_Hatter: I’ll share this with Shane. Feb 03 17:12
    ushimitsudoki Yup – 6 years and poof like that. Things getting rough – I was a gov’t contractor, so you know shit’s getting bad Feb 03 17:12
    schestowitz It gets bad with an MOU is signed. :-D Feb 03 17:13
    schestowitz ushimitsudoki: Japan, right? Feb 03 17:13
    ushimitsudoki Hah – making a site slashdot-proof is a good problem to have Feb 03 17:13
    ushimitsudoki Yes, japan Feb 03 17:13
    schestowitz I’ve been reading a fair bit on what happens there. Feb 03 17:13
    schestowitz Economy-wise, not IT–wise. Feb 03 17:13
    schestowitz ushimitsudoki: we stood stable before Feb 03 17:13
    schestowitz I use Coral Feb 03 17:14
    schestowitz But.. Feb 03 17:14
    schestowitz This means I need to be awake and aware when it happens Feb 03 17:14
    schestowitz Like an advance warning that some people E-mail us about. Feb 03 17:14
    schestowitz Then I can redirect to Coral. Feb 03 17:14
    The_Mad_Hatter advance warning of the slashdot effect? Sheesh. Feb 03 17:14
    schestowitz The issue is, people move b/w pages, so they kind of ‘escape’ the cashing sandbox Feb 03 17:14
    schestowitz *caching Feb 03 17:14
    *MinceR has quit (Remote closed the connection) Feb 03 17:15
    schestowitz The_Mad_Hatter: it’s easier with Digg Feb 03 17:15
    schestowitz You just put your domain and RSS Feb 03 17:15
    The_Mad_Hatter Just curious – how well backed up are you? Feb 03 17:15
    ushimitsudoki schestowitz: I wish I could make a good recommendation – but not my area. I thought I saw a wordpress plugin once that went to a static site automatically when a certain number of hits was reached? That’s one possibility Feb 03 17:15
    schestowitz Then you see if something advances close to FP and shield the pages up in the sense that you strip down the PHP file (trimming for DB load) or redirect Feb 03 17:15
    schestowitz The_Mad_Hatter: I back up regularly. So does Shane. Feb 03 17:16
    schestowitz ushimitsudoki: yes, WordPress though does what a server-level (not CMS-level) software achieves Feb 03 17:16
    The_Mad_Hatter What I meant was, is it possible to download the entire site? Feb 03 17:16
    schestowitz Also, we use the ‘LTS’ version of WordPress, so many plugins are likely not to be compatible Feb 03 17:17
    schestowitz The_Mad_Hatter: it used to be possible Feb 03 17:17
    schestowitz The host blocked it Feb 03 17:17
    schestowitz They set limits Feb 03 17:17
    schestowitz http://boycottnovell.com/site-archives/ Feb 03 17:17
    schestowitz IIRC, they took the site down for a few minutes, then put it back online with limits on queries. Feb 03 17:18
    The_Mad_Hatter Distributed backups. I have a copy of the Iowa MS stuff here. What I’m saying is that if you get knocked down, say MS sues you for libel, if someone else has a copy, well…. Feb 03 17:18
    schestowitz The_Mad_Hatter: they can’t sue for libel Feb 03 17:18
    The_Mad_Hatter I think you got the wrong host <G> Feb 03 17:18
    ushimitsudoki Whoa. Waugh left GNOME? Is there real hope for the GNOME project? Feb 03 17:18
    schestowitz And they won’t Feb 03 17:18
    schestowitz If anything, I could sue *Them* for libel if they chose this route Feb 03 17:19
    The_Mad_Hatter Of course they can sue for libel, or whatever else they dream of. They might now win, but they can cause as much trouble as possible. If someone else has a copy, well… Feb 03 17:19
    schestowitz I once spoke to a lawyer who got Microsoft convicted for libel. I spoke to him in 2006 Feb 03 17:19
    schestowitz ushimitsudoki: GNOME is fine Feb 03 17:19
    schestowitz I think Waugh was a PR disaster to them. He caused friction /inside/ GNOME. Feb 03 17:20
    schestowitz Murray for example went Rambo on him. Feb 03 17:20
    The_Mad_Hatter Hey, I’m paranoid. I’ve seen to much of this sort of stuff over the years. Feb 03 17:20
    *r0ver (n=r0ver@200.127.24.112) has joined #boycottnovell Feb 03 17:20
    ushimitsudoki schestowitz: I don’t trust it right now. Seems too many people are too eager to get mono all over it. With Icaza and Waugh out of the picture, things might look up, though Feb 03 17:20
    schestowitz The_Mad_Hatter: there are already 4 mirrors of Comes that I know of. Feb 03 17:20
    schestowitz It was a popular torrent in 2007 Feb 03 17:20
    *MinceR (n=mincer@unaffiliated/mincer) has joined #boycottnovell Feb 03 17:20
    The_Mad_Hatter Which is how I got it. And how many mirrors of BoycottNovell? Feb 03 17:21
    schestowitz ushimitsudoki: Mono is a separate issue. Feb 03 17:21
    schestowitz GNOME != Mono Feb 03 17:21
    schestowitz But GNOME is deployed with a lot of Mono Feb 03 17:21
    schestowitz Ubuntu contains Mono Feb 03 17:21
    schestowitz Ubuntu also contains GNOME Feb 03 17:21
    The_Mad_Hatter And I strip Mono out of Ubuntu. Feb 03 17:21
    schestowitz The_Mad_Hatter: no mirror of BN Feb 03 17:21
    ushimitsudoki schestowitz: In some ways, I agree. My point is a -lot- of people seem to be trying to change that into “gnome == mono” or at least “all the cool bits of gnome == mono” Feb 03 17:21
    The_Mad_Hatter There’s several threads in the Ubuntu forums covering the issue. Feb 03 17:22
    schestowitz I don’t think we would mind one. I could set up one at work (simple Apache+PHP+WP thing) Feb 03 17:22
    ushimitsudoki I felt like that was something Waugh was supportive of (the further integration of mono in gnome). And I -know- that’s something icaza tried to push Feb 03 17:22
    The_Mad_Hatter I think there needs to be mirrors of BN, preferably in different countries. Feb 03 17:22
    schestowitz The_Mad_Hatter: the DB is the core of it Feb 03 17:23
    The_Mad_Hatter Or at least archive copies, that can be put up quickly if anything happens. Feb 03 17:23
    schestowitz So it could be floated around without the passwords in the DB. Feb 03 17:23
    schestowitz Groklaw has a mirror too Feb 03 17:23
    schestowitz Our issue, however, is separate Feb 03 17:24
    The_Mad_Hatter Of Comes yes. Of the writing you’ve done, if the have it, I didn’t see it. Feb 03 17:24
    schestowitz Let’s say the load on “Effects” is not the issue but regular loads also Feb 03 17:24
    schestowitz About 90 DB queries/seconds Feb 03 17:24
    schestowitz I think Squid will improve this Feb 03 17:24
    schestowitz It ought to serve identical pages without rerunning the whole thing each time Feb 03 17:24
    The_Mad_Hatter I don’t know, that sort of stuff is beyond me. Feb 03 17:25
    schestowitz Since people don’t hammer on the archives as much as on new posts, that ought to help a lot Feb 03 17:25
    schestowitz I don’t know how Pound and Squid work because I never tried it Feb 03 17:25
    *r0ver (n=r0ver@200.127.24.112) has left #boycottnovell Feb 03 17:26
    schestowitz I imagine that it’s an abstraction layer that keeps copies of pages served and can bypass WordPress Feb 03 17:26
    The_Mad_Hatter Me either. Feb 03 17:26
    schestowitz In which case, this would help a lot and keep us on cheap hosting. Feb 03 17:26
    schestowitz Shane pays those bills. Feb 03 17:26
    schestowitz The ads help cover this Feb 03 17:26
    schestowitz So if we moved elsewhere, that would complicate things, I think. Feb 03 17:27
    The_Mad_Hatter Heh. Thinking about off-site backups, why setup a torrent? Feb 03 17:27
    schestowitz “Why setup” or “why /not/”? Feb 03 17:27
    schestowitz The_Mad_Hatter: it’s not simple Feb 03 17:27
    The_Mad_Hatter Sorry, pain killers are kicking in. Why not? Feb 03 17:27
    schestowitz I need to ask Shane if he’s OK with floating his posts and the passwords need to be stripped off the DB Feb 03 17:28
    The_Mad_Hatter I meant technically. Feb 03 17:28
    schestowitz The_Mad_Hatter:  I have statics of the pages. I could zip them up and make that a torrent hosted from here. Feb 03 17:28
    schestowitz That would make the equivalent of paper trail ‘backup’ of the site Feb 03 17:29
    The_Mad_Hatter It’s possible to set up a torrent, and add to it over time. Feb 03 17:29
    schestowitz But anyway, that’s a separate matter Feb 03 17:29
    schestowitz Another issue is navigation, I think. Feb 03 17:29
    The_Mad_Hatter So once a week you add the new pages, and anyone who is downloading the torrent gets them automatically. Feb 03 17:29
    schestowitz If you want to make a sort of ebook, that mat be possible, but it need grepping Feb 03 17:29
    schestowitz Like s/http:\/boycottnovell.com\/./ Feb 03 17:30
    The_Mad_Hatter yes, that could be a bit of work. Feb 03 17:30
    schestowitz Oops, sed, not grep Feb 03 17:30
    schestowitz Anyway, regarding hosting, it’s up to Shane really. Feb 03 17:31
    schestowitz I’ll post that log for him to read. Feb 03 17:31
    The_Mad_Hatter I’ve tried setting up an apache/php server once or twice, and messed it up pretty badly. Feb 03 17:31
    schestowitz That’s acceptable for just secondary (backup) Feb 03 17:31
    The_Mad_Hatter K, anyway I better go grab another pill, I’m hurting like a bastard today. Feb 03 17:31
    jose__ The_Mad_Hatter, have you added to existing torrents before.. i thought that affected the torrent attributes and effectively means you are creating a new torrent .. meaning, i would imagine, that it would not be picked up by those grabbing the old (???) Feb 03 17:31

  2. mike said,

    February 3, 2009 at 2:47 pm

    Gravatar

    Haven’t dealt with this sort of stuff since before ’slashdot effects’ even existed …

    Possible relatively low-tech solution – redirect the front-page – no doubt the busiest/heaviest to make – to a static page, generated every few minutes/less often if busy, or when something on it changes. e.g. through the web-server rules + cron.

    Not that this would address the problem expressed in the mail – too many http connections and too much bandwidth consumed.

    Sounds like wordpress doesn’t scale too well anyway with the no of queries going on. An alternative cms? (hey according to the IRC logs yesterday, open sauce == no lock-in – right!! ;-P).

    To be honest, in this day-and-age of slashdot-effects i’m pretty surprised it doesn’t do anything like this already. But then again, it is written in a scripting language (I worked on a php project many years ago, and have an intense distrust of php apps in general, about as much as say, vb apps).

  3. amd-linux said,

    February 3, 2009 at 3:01 pm

    Gravatar

    Roy,

    did you enable WP Cache? I guess so, but just to make sure….

    http://www.gunnart.de/tipps-und-tricks/blog-tuning-teil2-wordpress-cache-nutzen/

    This can reduce the DB load by 90%.

    WP-Super-Cache also comes to mind: http://wordpress.org/extend/plugins/wp-super-cache/

    You maybe can also consider removing links to the archives and the categorie cloud.

    If somebody wants to artificially increase DB load, these links are an invitation.

    Sorry if somebody came already up with these suggestions, had not time to check IRC logs.

  4. amd-linux said,

    February 3, 2009 at 3:37 pm

    Gravatar

    Forgot to mentioned that my blog ran quite well while on a IBM Thinkpad 600E (Ubuntu Dapper, Wordpress with WP cache plugin – with up 1500 unique visitors a day. That is much less than your 700k, but also check the specs of a Thinkpad 600E….

    http://www-307.ibm.com/pc/support/site.wss/document.do?sitestyle=lenovo&lndocid=AGRN-446MWD

    Not sooo bad for such a machine :-)

    So if your machine and setup would really scale well, 700k should be possible with 4 cores machine if my math is right.

What Else is New

  1. Links 18/3/2010: Steam and Linux; Red Hat's CEO Talks

    Links for the day

  2. IRC: #boycottnovell @ FreeNode: March 18th, 2010

    IRC Log for March 18th, 2010

  3. Former Microsoft Employees and Boosters Call Microsoft MVP Miguel de Icaza and Other Microsoft Apologists “Most Powerful Voices” in Open Source

    Microsoft folks have decided on 'our behalf' who is important to Open Source and who is not

  4. Magalhães + Microsoft = Corruption

    Microsoft accused of blocking GNU/Linux and more leaks about this scandal are high in demand

  5. Open Irony: Microsoft Creates/Sponsors OpenMainframe.org to Attack GNU/Linux

    War is peace and Microsoft is the new "open"; Details on the latest attack of Microsoft against GNU/Linux, using proxies

  6. Microsoft Brings MPEG-LA-LA Land to the Web and Threatens GNU/Linux With Software Patent Lawsuits

    Microsoft is trying to sneak patents-encumbered MPEG formats into the Web using Internet Explorer 9 (IE 9); Microsoft threatens (again) to go after Linux legally

  7. IMAX -- Not Just Apple -- Attacks Free Software With Software Patents

    Another legal attack against Free software comes in the form of a threat (issued against Sandy3D) and Apple's reason for suing Android seems like gradual iPhone defeat (Linux is winning)

  8. Links 18/3/2010: Many IBM Headlines, Mandriva Enterprise Server 5.1

    Links for the day

  9. IRC: #boycottnovell @ FreeNode: March 17th, 2010

    IRC Log for March 17th, 2010

  10. Microsoft -- Like Gates Foundation -- Still Uses Own 'Studies' for PR and Lobbying Purposes

    Some of Microsoft's latest 'studies' (from the past week) are looked upon more closely for their true purpose to be understood

  11. Microsoft Entryism Roundup: COPsync, Level 3, and Yahoo!

    COPsync hires from Microsoft, Level 3 dumps Microsoft's proprietary software to stave off Free software, and Yahoo! keeps falling apart

  12. Internet Explorer 9 Seems Less Secure Than Predecessors; Microsoft Plays the Vapourware Game Against Rival Web Browsers

    Internet Explorer 9 removes security features and lies about its standards compliance using improper benchmarks

  13. More Evidence of Potential Microsoft Involvement in Apple-HTC Lawsuit Against Linux/Android (and Microsoft Loses to Virnetx)

    Microsoft's top "IP" bullies commend Apple's legal action and Microsoft owes VirnetX $105.75 million for patent violation

  14. Novell Wants to Bring Microsoft, Moonlight, and Mono to Linux Phones (Android)

    Microsoft's patent-encumbered 'gifts' to GNU/Linux are being pushed into devices with Novell's help

  15. Patents (on Life) Roundup: Human Tissue and Crops Monopolised

    New reminders of the dangers of an excessively broad patent law

  16. Links 17/3/2010: KDE 4.5 Proposals, Benchmark of Distros in Development

    Links for the day

  17. LCA 2010 Conference - Glyn Moody Keynote

    Just a video

  18. Gates Roundup: Monsanto, Colonisation, Clinton Ties, and Control of Education

    Rich uncle Bill, who is said to be giving away his massive wealth, has somehow made $13 billion over the past year

  19. Does Microsoft Tinker With the Search Bar in Firefox?

    A reader alleges that Microsoft is playing dirty on Windows in order to suppress the use of Google (assuming AVG agreed to reroute traffic to Microsoft via Yahoo!)

  20. IRC: #boycottnovell @ FreeNode: March 16th, 2010

    IRC Log for March 16th, 2010

  21. Quebec Authorities Should be Sued Again for Microsoft Corruption; BECTA Should Too

    Quebec's government is up to no good again (bidless procurement suspected); the time is right to challenge BECTA legally, just like in Quebec

  22. ACTA Booster Luc Pierre Devigne Redefines Open Standards (With Software Patents Included)

    The European Commission turns its back on open standards that anyone can accept; ACTA and the Digital Economy Bill show a legal land grab by corporations

  23. “Call Out Windows”

    Prelude to a new campaign which strives to change the coverage of Windows-specific security problems

  24. Microsoft Happy About Apple's Invocation of Software Patents Against GNU/Linux

    Mobile Linux proceeds from one victory to another while Apple and Microsoft pointlessly attack with software patents

  25. Links 16/3/2010: Amarok 2.3.0, SimplyMEPIS 8.5 Preview

    Links for the day

  26. Microsoft Bing in the Business of Deceit, Censorship, and Brainwash

    Microsoft's idea of "search" continues to incorporate business bias, unnecessary censorship, bribes, and advertisements that disparage Google

  27. Government Should Avoid Proprietary Software and 'Clouds'

    Reporters wonder if Vivek Kundra (national CIO) can resist the temptation to just hand over government operations to private companies, some of which are abusive and dangerous to sovereignty

  28. Microsoft 'Pulls a SCO' in India (Against GNU/Linux)

    Poor Microsoft complains about a "mainframe monopoly" which does not run Microsoft Windows and the same strategies it used in Europe are being extended to India

  29. BSA Blames Lack of Patents for Europe Being 'Behind', EPO Loses €2 Million in Kaupthing Bankruptcy

    More brainwash from Microsoft's front group, the BSA (with former employees of Gates Senior); Europe's patent office -- not Europe itself -- may be having problems

  30. Mono Influence Increases in the GNOME Foundation

    Incorporation of Microsoft into GNU/Linux (especially GNOME) continues unabated

RSS 64x64RSS Feed: subscribe to the RSS feed for regular updates

Home iconSite Wiki: You can improve this site by helping the extension of the site's content

Chat iconIRC Channel: Come and chat with us in real time

Recent Posts