Showing posts with label ruby on rails. Show all posts
Showing posts with label ruby on rails. Show all posts

Tuesday, April 22, 2008

Aptana - Please Don't Suck Any Harder

I wrote this post after hours of frustration, trying to fix my broken environment, and losing valuable time on a project that was falling behind.

I now realize that the post was overly emotional, and probably not entirely fair to the Aptana team. I'll post updates here as I work through the issues to get my RadRails working again.

Tuesday, October 16, 2007

Ruby on Rails Hosting: From HostingRails to RailsMachine in a shake of a tail

Rails hosting is a hot subject, and with everyone asking everyone else about their experience, I thought its only fair to share my own experience, even though it is relatively limited.

Introduction

I started with a shared "professional" hosting plan from HostingRails. It was around $30/month, and included non-root SSH access to a shared server, and additional 150Mb of RAM for total of 200Mb (although that's actually quite misleading, see below). I stayed on this plan for about 3-4 months, and then decided to switch to a virtual hosting plan from RailsMachine - their single server plan, for about $100/month (that includes dedicated 384Mb of RAM and a root access to my virtual server). This post describes the reasons behind switching, and compares pros and cons of each hosting plan. Our needs included the following setup:

  • One application running in two instances (one production, and one test)
  • Two-mongrel instances per application (so total of 4 mongrels)
  • Capistrano based deployment / upgrades / restarts of the servers
  • Apache 2+
  • PostgreSQL 8.2
  • Trac (+ MySQL) served over https
  • Subversion repository served over https
  • Subversion post-commit hook linked to a build environment, for continuous integration
  • Automated remote daily backup of the entire environment

To be fair, the plans described here compare sort of like apples and oranges, so the review has to be taken with a grain of salt. Main differences are, of course, in the fact that one is a shared plan and the other one virtual hosting. People on smaller budget and without tight performance requirements will probably prefer the shared plan, while those with a bit more room in the budget and some UNIX know-how will most likely be much happier with virtual hosting. My hope is that perhaps this review will help someone make the right call for their environment based on their needs, budget and experience.

Detailed Comparison

1. HostingRails

  • Pros
    • Cheap, only $30/mo for 4 mongrel instances (200Mb)
    • Easy Trac/SVN install
    • Rudimentary backup can be run from web UI
    • Friendly support staff
    • Timely support, usually within an hour or two
    • Lots of info on forums
  • Cons
    • [Update: please see comments section for HostingRails response]. Slow, slow, and slow. We were sharing a server with 4Gb of RAM with another 50+ mongrel instances run by as many users. The server used over 1Gb of swap space and often showed load averages over 30. We were promised numerous upgrades, which did not materialize while we were using the service. Our sites and Trac at times took over 30 seconds to load the first page, probably while the corresponding process was getting un-swapped.
    • Have to email support for any changes to top-level apache configuration, such as adding domain aliases. This applies to setting up Trac, svn and mongrel apps.
    • Provided backup facility does not backup PostgreSQL/mysql DB, had to write our own script (but forums had a good starting point)
    • Our mongrels clearly were not allocated any amount of "reserved" RAM. Looks like buying extra 150Mb in reality means you can run three more mongrel instances, but has nothing to do with how much RAM you actually get, as was witnessed by swap usage stats.
    • As we decided to move because of performance issues, HostingRails refused to refund us the unused portion of the yearly fee we paid ahead as we were past first 30 days.
    • Locked in their choice of Ruby version, gems, PostgreSQL, MySQL, Trac - can not upgrade these individually, as each is shared among many clients.
Overall, I think that HostingRails has to fix performance issues and not pack as many clients on a single server if they want to be a viable solution for any business. Until they fix performance issues I would NOT recommend this service to anyone. If the performance is fixed, I would recommend this plan to less experienced engineers, who may not know how to use root on a UNIX system, or setup Apache, but have some basic understanding of Rails deployment. I would also recommend this plan on those with a really tight budget.

RailsMachine

RailsMachine provides virtual hosting, meaning you get your own dedicated server with root access. The server, OS and pre-installed software details can be found on RailsMachine website. Because you are sort of expected to use root access, some basic UNIX system administration skills are necessary to take advantage of this plan. However, the flexibility and power that come with this solution by far outweigh the additional $60/month we are paying for our own box. Note that we could pay only $75/month for 256Mb of RAM, but considering the list of applications we wanted to run, I chose to upgrade our plan to at least 384Mb.

  • Pros
    • Much, much, much, faster. When using command line, sometimes you see server "hanging" momentarily while presumably other virtual machines are being processed, but the overall performance improvements are quite dramatic. Pages (trac/mongrel) load within 1-2 seconds consistently.
    • Fantastic provided Apache configuration, comes with plenty of dynamically compiled modules, template config files and modular structure. Our single apache server is providing umbrella access to both mongrel apps, SVN over https and Trac over https (using mod_wsgi, not CGI).
    • RailsMachine gem allows seamless initial capistrano deployment and subsequent installations
    • Apparently each virtual server is backed up automatically (but we do our remote backups anyway)
    • CentOS Linux provides convenient yum utility for installing pre-built packages. Sometimes it takes a few minutes to find the right package using "yum search" but once found, installation is a breeze.
    • Root access allows installing what you need for your environment, eg - latest PostgreSQL, latest Trac, etc. Was able to install mod_wsgi module to enable much faster Trac access than using traditional CGI access.
    • Our entire setup described above takes up 180Mb of RAM, with another 180Mb left for file system caching. Pre-installed MySQL and Apache are built with multi-threading enabled, so the processes manage their shared RAM very effectively. This means we have room to add another 2-3 mongrel instances to the setup without affecting pricing of our plan.
    • Responsive, knowledgeable support, although just the nature of the setup means you have to ask a lot LESS questions if you know what you are doing, since you can get most of the things installed yourself without bugging them.
  • Cons
    • [Update: this has been fixed, see comments section] The server came installed with broken permissions on /etc/resolv.conf and /etc/hosts file. This broke host/domain name resolution from the host itself, until the permissions were set to allow read access to the above files.
    • Slightly more expensive.
    • Occasionally (but mostly rarely), shell access has a latency that appears to be CPU bound by the physical server and not the network.
Overall, I would absolutely recommend RailsMachine to a technically savvy engineers, who want freedom of their own box at a very affordable price. Ability to grow the size of the virtual box dynamically is also great - it's very nice to know that when you need extra 1Gb of RAM, it's there waiting for you (and your wallet).

Install Notes

A quick note about SVN/Apache configuration with authorization turned on. This has been mentioned on many SVN forums, but if you are setting it up, please be aware that mod_dav_svn contains memory leak that makes apache seg-fault during checkouts of a large repository tree (we are using authorization as well as authentication). I had to add the following clause to apache config for SVN directory to resolve to issue: SVNPathAuthz off This is what our SVN configuraiton looks like in case someone is looking for a complete config:

<VirtualHost *:443>
 ServerName svn.YOURDOMAIN.com

 SSLEngine On
 SSLCertificateFile /etc/httpd/sslcert/server.crt
 SSLCertificateKeyFile /etc/httpd/sslcert/server.key

 <Location />
       DAV svn
       SVNPath /var/lib/subversion
       AuthzSVNAccessFile /var/lib/subversion/conf/authz
       SVNPathAuthz off

       AuthType Basic
       AuthName "Authorization Realm"
       AuthUserFile /var/lib/subversion/conf/passwd
       Require valid-user
 </Location>
</VirtualHost>

References

  1. Memory Leak in mod_dav_svn

Closing Notes

Here are the responses I got from HostingRails and RailsMachine - thanks guys...

Hi Konstantin, The issue with broken permissions on hosts/resolv/others has been fixed. It was due to a small (and short-lived) bug in our provisioning system. Thanks for mentioning us! -Rob at Rails Machine
Hi Konstantin, there’s a couple things I’d like to point out that I hope you’d be willing to correct in your review. I understand that you had a bad experience with us and respect your right to voice it.
  1. Running a cluster of 4 Mongrels will only cost $22/mo on our professional plan, and $26, $32/mo on our business and platinum plans, respectively.
  2. On our site we outline that we’re selling (physical + virtual) memory in blocks of 50MB on shared servers — when Mongrels or Static FastCGI instances are fired up they consume (’reserve’) these resources persistently.
Overall, and I suppose you’d have to take my word for it, your server ran into an unfortunate resource situation with a number of users who are now either suspended due to malicious behavior or upgraded to dedicated environments due to extraordinary growth of their apps. Most of our clients are having good hosting experiences with us and we’re working hard to maintain fast and reliable shared servers for Rails deployment at the lowest possible price. All the best, ~William at HostingRails

-- William, thank you for your corrections and I appreciate that it must not be easy to maintain quality service at such a competitive price. I’ve updated the pricing I mention and added a reference to your comment. I am willing to believe that our server may have been exception rather than a rule, and I hope that in the future you have a way to move clients off loaded machines quickly.

Tuesday, September 25, 2007

On Ruby on Rails with PostgreSQL, and Acts as Paranoid

Back a few years ago I was researching differences between PostgreSQL and MySQL databases, and chose PostgreSQL because at the time it supported foreign key constraints and many other fantastic SQL extensions that make developer's life a lot easier. Today I am sure MySQL is just as functional as PostgreSQL, and it does appear to be a more popular choice as the Rails DB than MySQL. I still prefer PostgreSQL, it just feels more natural to me coming out of Oracle background, so I am probably biased (who isn't?) Anyway, in the last year and a half or so, I've been writing Rails apps that use PG-based databases and found a few random tricks I'd love to share here. Opinions differ here, but just like accountants like the ancient double entry accounting system, I personally prefer a double validation system - where Rails validates my objects before/after updates, but the database double checks this using proper constraints. Rail's validation system is very robust and extensive, and it is its power. However nobody is protected from human error, and it's pretty easy to forget certain validations, some validations may be performed conditionally, and in general rails validations are at a higher level. I've certainly missed a validation or two in the past myself, or had a bug in my condition. So as a rule, I specifically like to use in all of my migrations:

  • Foreign key constraints, with "ON DELETE CASCADE" to allow tests to drop/remove fixture data
  • Check constraints to verify values satisfy certain conditions. Use these to validate finite value columns, such as state/status, gender, inheritance discriminator column, or polymorphic table's type discriminator. Use it to validate pattern based columns, eg if you decide to store SSN as a string of the form XXX-XX-XXXX, a check constraint can ensure it really is.

Foreign Key Constraints

Let's start with foreign key constraints. They are extremely useful in ensuring your relationships work and do not point to non-existing rows. It is even more important if you bypass ActiveRecord and use direct SQL in certain cases to insert data (as this may be the case with some batched or high-volume data loads). I use foreign keys on ALL of my tables, and haven't had much trouble with that. The only issue to watch out for, is the order in which your fixtures are loaded. Basically you'd want to load your independent tables first, such as countries/states - those that do not depend on anything, and then load fixtures that depend on them in the order of dependence.

Please see the following post for an opposite opinion, and note that I don't agree with author's suggestion not to use foreign keys - read first comment, I too was saved on multiple occasions by having them, especially in the development (since that's where you'll be finding most bugs in the first place).

Here's how to add a foreign key constraint in your migration:

create_table :profiles do |t|
   t.column :user_id,    :integer, :null => false
end
execute "ALTER TABLE profiles 
          ADD CONSTRAINT profiles_fk_user_id
          FOREIGN KEY (user_id) 
          REFERENCES users (id) 
          ON DELETE CASCADE"

Notice how all constraints are named. This is very important, because you want to be able to change/update constraints in the future (especially the check constraints - see below). Not giving a constraint a name explicitly forces DB to come up with an auto-generated name, which is not very useful. Any future migration that needs to change this constraint would be at loss as to how to reference it reliably. Using a proper naming convention is also a good idea: table_fk_field is common naming practice for foreign key constraints.

Check Constraints

Check constraints verify that values in a column satisfy provided conditions. They are extremely useful in ensuring you don't get "^%$$$" as your gender value for user Bob, or "Frak!" as a status for your order. Database can be manipulated in many ways, and in my experience unless you protect your columns some weird stuff always ends up in there, whatever the greatest validation framework sits in front of it. In this hypothetical example, we need to ensure that the gender column on our frogs table only allows "M", "F" or "T" as possible gender values (I live in San Francisco, yo!). So in the migration file, specify check constraint similar to how we did this for foreign key constraints:

create_table :frogs do |t|
   t.column :name, :string
   t.column :gender, :string,   :limit => 1, :null => false
end

execute "ALTER TABLE frogs 
          ADD CONSTRAINT frogs_check_gender 
          CHECK (gender IN ('M', 'F', 'T'))"

Please see the following link for more information on PostgreSQL powerful constraints syntax: PostgreSQL Constraints

Partial Indexes, and Acts as Paranoid

If you had to install acts_as_paranoid plugin, you'd notice that it adds deleted_at column which is going to be NULL for all values that are active in your database. Calling destroy will not physically delete the record, but would merely set deleted_at to the current time stamp. By declaring a model as "acts_as_paranoid" it will also add the following to all queries: and (deleted_at is NULL or deleted_at > ?) First of all, the "deleted_at > ?" clause is meant for allowing models to expire in the future, but I haven't met a single person who uses this feature. So the first thing I do when I install this plugin is I modify it's source to remove this comparison. I want my find statements generate the following: and (deleted_at is NULL) Ok, so things are great now, but suddenly I realize that my favorite index on users table and email field, does not really work as well! But of course... I am now querying on both "email = ? and deleted_at is null". So how do you add deleted_at to the index? Here's how:

execute "CREATE INDEX users_idx_email 
            ON users (email, deleted_at) 
            WHERE deleted_at IS NULL"

This uses a cool feature of PostgreSQL called partial index to create an index on subset of values.

Comments and thoughts/suggestions are always welcome! References:

  1. http://www.oreillynet.com/ruby/blog/2005/11/migrating_to_ruby_on_rails_and.html
  2. http://www.postgresql.org/docs/current/static/index.html

Friday, September 7, 2007

Ruby on Rails: Reducing clutter in actions by placing common code in filters

This is a tiny but useful tip, that saved many lines of repeated code in my controllers, hence why not share it :)

Do Not Repeat Thyself?

If you've looked at the controller code that's generated by scaffolding, you'll find something like this:

 
  def show
    @user = User.find(params[:id])
  end

  def new
    @user = User.new
  end

  def destroy 
    User.find(params[:id]).destroy
  end

Sure, in this case all we are doing repeatedly is instantiating the user by a potentially available parameter value. What if instead we placed these common fetches in a controller filter, which would simply set an instance variable for us? Hell, we could even handle exceptions (such as invalid ID) in only one place this way! What not to like.

When dealing with a more complicated route that has been defined, this instantiation may become quite a bit more elaborate, and the case for a filter is even more justified.

Consider the case of building a collaboration system where you have projects and individual contributions under that project, as well as a producer of the project. We might want to support all project operations under a URL that looks kind of like this:

/user/kigster/projects/MyVacation/contribution/view/34 with a corresponding route in the config/routes.rb file:
# project management route
map.connect '/user/:username/project/:project_name/contribution/:action/:id',
    :controller     => "contribution"

Based on the route defined, Rails would create params[:username], params[:project_name] and params[:id] for contribution id, in addition to the standard action/controller pair.

Now imagine that the controller we are writing has many actions, such as add, edit, view, list, append, preview, post, comment, etc. All of them could use a handle on the project, it's producer and contribution instances. Ideally - in @project, @producer_user and @contribution respectively.

If we followed the scaffolding example, we'd simply add appropriate lookups at the beginning of each action. But that's a lot of repeated code!

So let's use filters instead, to get our common lookups under control.

class ContributeController < ApplicationController
  before_filter :setup_project

  def home
     # do stuff
  end

  private

  def setup_project
    @producer_user = 
      User.find_by_username(params[:username])
    @project = 
      @producer_user.produced_projects.detect do |p|
      p.name.downcase.eql?(params[:project_name].downcase)
    end
    @contribution = 
        @project.contributions.find_by_id(params[:id])

    return true

  rescue Exception => e
    logger.info "can't show project:", e
    flash[:error] = e.message if e
    redirect_to :controller => "home" and return
  end
end    

This is short and elegant, and now every action in this controller (which is not excluded for setup_project filter) will have access to our instance variables. Groovy!

Issues

Of course nothing is free, and in this case we are potentially loosing performance. Perhaps some actions don't need to know the @producer_user. We'll be fetching it all the time, which saves time for us - developers. If performance problems occur because of extra unused fetches the filter can be broken up or optimized. Until then - it sure saves me a headache.

That's it!

Thoughts, comments?

Wednesday, December 6, 2006

Ruby on Rails: How to load session objects into console

This took a little bit of poking around, so I decided to post it here.

Basically I was tracking down a bug, which resulted in a corrupt object (cart) in the web session. So I thought, it would be nice to play with this object in the console to see what's up... But how do I get it in there?

There may be a more elegant way to do this but here is how I got it to load:
  1. Look in tmp/sessions and find the most recent file (ls -alrt on UNIX). Let's say the file is called 'tmp/sessions/ruby_sess.8eb9614a7e4e1e3b'
  2. Open console and type:
    >> session = Marshal.load(File.open('tmp/sessions/ruby_sess.8eb9614a7e4e1e3b'))
    => {"hash"=>{:cart=...
    >> cart = session["hash"][:cart]
    ....
    
    In this case I was trying to access a cart object in the session, which was placed in the session with:session[:cart] = Cart.new
That's it!