Friday, April 11, 2014

Wanelo Tech Gems: we've been busy!

It's been quite some time that I've posted here, to my personal blog, so long in fact, that the publishing interface a'la Blogger now looks like MS Word. Or God forbid you remember, Word Perfect. (Yes, I am that old).

But that's not what I wanted to write about.

Wanelo, the team that I lead as a CTO, has been kicking so much ass lately, that I've just had no time documenting personal projects here, even though there have been plenty.

So to sort of catch up in one blog post at once, I wanted to put a few links to some of the great content on our technical blog, that's been recently migrated to Github Pages.

Without further ado:

Multi-process or multi-threaded design for Ruby daemons

A pretty awesome blog post by our own Eric Saxby about thinking in terms of GIL (global interpreter lock in ruby), it's impact on production environments, especially in multi-threaded environments, and where long-running processes may have impact on your database transactions. Very worthy read.

A Brief History of Sprout Wrap

This great post by James Hart talks about our usage of Sprout-Wrap to automate developer station setup, and turn it into a pretty much a four-five command process, and about an hour of automatic magic.

12-Step Program for Scaling Web Applications on PostgreSQL

By yours truly, is a deep dive into performance journey of Wanelo.com into the world of multiple thousands of requests per second, light-weight HTTP services, horizontal and vertical sharding, as well as a slew of useful tips and tricks along the way.

Finally, and most recently, 

Capistrano 3, You've Changed! (Since Version 2)

James published another detailed gem about upgrading to Capistrano 3, and iterates over some of the most interesting changes in tasks, locations, command line, rollbacks and more. If you do deployment with Capistrano you need to read this.

I'm going to do my best to keep up with what's going on, and keep this up to date. 

Perhaps I'll also change the template, so it's not as ugly to look at. LOL.

Till next time, signing off. –– KG.


Tuesday, August 6, 2013

Wanelo Blog Post: Detangling Business Logic in Rails Apps with PORO Events and Observers

I recently wrote a blog post about how Wanelo uses events and observers in plain ruby, to model application events. The base code that supports it is available in the open sourced gem ventable.

If you want your business logic "glue" to look like this below, read the above blog post.
Wanelo::Event::ProductSave.configure do
  notifies Product,
           SaveNotification,
           SaveAction,
           inside: :transaction

  notifies ProductSaveWorker,
           ResaveEmailWorker
end
We have found so many applications of using events in the backend Rails application, that I can not imagine building another app without it.

Having said that, right now Ventable does not support distributed events. But it absolutely could! I am considering writing a plugin for sidekiq background processing framework, that could notify any observer via a background job, and not inside the current ruby thread.    Another option is to use another thread! Lots of options for concurrency.

Perhaps one day, it will look like this:
Wanelo::Event::ProductSave.configure do
  notifies Product,
           SaveNotification,
           SaveAction,
           inside: :transaction

  notifies PostSaveProcessor, via: :worker
  notifies ResaveEmailWorker, via: :thread
end

Friday, September 14, 2012

Wanelo Tech Blog: The Big Switch: How We Rebuilt Wanelo from Scratch and Lived to Tell About It

Just published my first contribution to Wanelo Blog on the subject of rewriting technology stack from scratch, and moving from Java to Ruby.

Feel free to leave comments here also!

-- Konstantin (at GoGaRuCo 2012).

Wednesday, August 8, 2012

Nagios checks for Sidekiq Queue size monitoring and Joyent RAM size

At Wanelo we are committed to sharing as much of our code as possible with the world, since we are ourselves highly reliant on open source movement (ruby, rails, etc).

Here is a small but growing project with simple BASH shell scripts used by Nagios monitoring system to ensure things are running smoothly.

https://github.com/wanelo/nagios-checks

Please see the examples in the README on how to use it with nagios.

Thursday, July 26, 2012

Getting RMagic and friends to work on OS-X Mountain Lion

Upgraded my ruby environment today to Mountain Lion.

Here is a quick checklist that I went through to get everything working.  The largest change was having to reinstall XCode and command line tools, and also download XQuarts in order to reinstall ImageMagick successfully. Without it, I was getting errors building RMagick of the following shape:

ld: file not found: /usr/lib/libltdl.7.dylib for architecture x86_64clang:
error: linker command failed with exit code 1
(use -v to see invocation)
make: *** [RMagick2.bundle] Error 1


Quick checklist:
  1. Install Mountain Lion
  2. Install XCode 4.4
  3. Install command line tools from XCode 4.4 Preferences dialog
  4. Install XQuartz
  5. In terminal run
    1. brew update
    2. brew uninstall imagemagick
    3. brew install --fresh imagemagick
    4. wipe out your ~/.rvm folder
    5. reinstall RVM and install the latest ruby 1.9.3-p-194
  6. run "bundle" in the project folder
  7. run "rake" and rejoice

References:

https://github.com/mroth/lolcommits/issues/65

    Wednesday, July 18, 2012

    Activity Feed Design

    I normally don't publish work diagrams, but this one sort of looks like work of art.

    It represents a collective design we came up with around aggregation of activity feeds on wanelo.com. If you have an account, you will soon see the results of this on http://wanelo.com/following


    Thursday, July 12, 2012

    Using CarrierWave with a dynamic fog host and a CDN

    Another simple case of "this took longer than it takes to look up this issue on Google" :)

    CarrierWave is a fantastic and well supported ruby gem for managing images, including support for cloud storage like Amazon.

    Whether or not you use CarrierWave on the cloud, you almost always want to put your user-uploaded images behind a CDN.  There are many CDNs available, including Amazon's CloudFront, Fastly.com, CacheFly, Akamai, etc, and comparing them is outside the scope of this article. 

    Because browsers, especially older, are limited in how many connections they will establish to a single host, it is sometimes beneficial to have images load from several alternating URLs, for example:
    http://cdn-0.wanelo.com
    http://cdn-1.wanelo.com
    http://cdn-2.wanelo.com

    etc...

    CarrierWave supports fog_host variable which can be set to either a string (a static hostname), or a lambda (if for example a randomized string is desired) and the usage is well described here.

    Unfortunately, due a bug that is not yet fixed (and unclear if it will be), you currently can not use the file object directly inside the lambda as it's shown in the examples. 

    If you actually want to use the "file" object inside the fog_host proc in any way, then fog_host must be a double-lambda, because the first lambda gets called by the accessor created in Configuration class. The second is then called in public_url instance method.

    Here is the configuration we ended up using, which works perfectly:
        config.fog_host = lambda do
          lambda do |file|
            host_number = file.path.sum % 4
            "cdn-s3-%d.ourdomain.com" % host_number
          end
        end
    
    The advantage of this approach, is that with several CDN hosts (which are useful to increase speed of browser's parallel download of images), this allows you to generate a consistent URL for each file instead of a random one. So a file with name "image.jpg" will always generate "cdn-s3-2.ourdomain.com" because of simple assignment:
    "image.jpg".sum % 4
    => 2
    Thanks to Jay Phillips for the tricky solution to this one.

    Sunday, June 10, 2012

    MagmaRails, and Enterprise Architecture with Rails

    This year's MagmaRails, where I was cordially invited to speak alongside Dr Nic, Aaron Patterson, Ron Evans, Blake Mizerany, Les HillDaniel Fischer and many more talented speakers from Mexico and USA, went without a hitch in a beautiful Manzanillo Municipality, Colima, Mexico.

    Organized by Crowd Interactive, this was by far one of my favorite conferences based on the amount of stuff learned, quality of personal interactions, general level of enjoyment, and also, of course, the drinking that went on with all the other fellow attendees and the speakers.  What a blast! Can't wait for the next one.

    Meanwhile, I've revised my presentation that was originally conceived for SFROR meetup, with more clarity, and a whole new section on how to get things started up and move fast, based in part on an ongoing professional experience at Wanelo.com.

    Here is the presentation!  Please leave comments for any feedback or suggestions.

    View more presentations from Konstantin Gredeskoul

    Sunday, June 3, 2012

    Starting a modern agile rails 3 project for a scale, from scratch - Part 0


    As I am entering my fifth month working at Wanelo, I feel very fortunate to be working with an amazing group of creative and compassionate people, building an awesome new application for the new generation social shopping platform that Wanelo had become.

    Because it is all so new — the team, infrastructure, the new application, we had to quickly make a lot of fundamental decisions about what toolsets to use, what processes to follow, how to optimally develop the new code, and how to build features as quickly as possible, how to minimize the amount of technical debt, and how not to forget about the scale (we have plenty of traffic already). Most of all, we wanted to have a ton of fun along the way :)

    The stack

    For our technology stack we chose Rails 3.2.3, Ruby 1.9.3 and PostgreSQL 9.1 as our base choices for the application. Besides these very fundamental choices, there are easily hundreds of tiny everyday decisions that a team on a new rails project needs to make. Once the basics are figured out, the frequency of these "big" decisions goes down. But it never stops, as the software for an active application is rarely frozen.

    A bit of history

    My very first Rails project was back in 2006, and it was an e-commerce site. There was just one magic book back then, and that was part of the attraction of Rails. Blurb.com launched a custom build e-commerce site in 4 months with 4 engineers, of whom all but one were new to ruby and rails. Having come from a recent Java project, I was blown away by how quickly everyone had become so incredibly productive.

    Since then, attracted by the same promise, thousands of developers had joined, and the ruby universe exploded. What's interesting, is that initially, and importantly, this technology attracted some of the brightest minds in the tech industry willing to escape to the world of software built in a cleverer, more coincise way that is both maintainable, incredibly compact, and follows the best patterns of software development, such as automated testing (or even TDD). Escaping from Java or PHP to Ruby brought happiness to people's career, including myself.

    As a result of exploding ruby-verse, the active toolset changes often. Any new team, having chosen their base set of tools, would have probably chosen a different set six month earlier. This is a unique trait of our chosen beloved development environment — i.e. building enterprise and web software with ruby (and often rails), and navigating it's vibrant, ever changing software ecosystem.

    There are many excellent (and often free) resources of information on the various open source options available to solve common problems. And when building a web app, especially starting it, many problems you encounter are very common and probably already solved. So,

    What tool is appropriate for ___ task?

    Should I use a gem or roll my own code for _____?

    How do you setup continuous integration?

    How should we all communicate?

    Should we be cowboy programmers and work individually "in corners" for 16 hours, or should we use paired programming?

    These are the many questions the new team must answer, and quickly. Arguing about these things is wasteful for the business. Even without an argument, each choice takes time.

    Some resources

    There is no shortage of most excellent resources (some free, some minimal and well deserved fee), such as:
    And watching/listening to these is most definitely one of the best ways to learn what's out there, and pick up some of best practices along the way. 

    But that takes time, and time is often scarce. So for those of you really needing to take a shortcut and kick start a modern rails 3 application, follow this series and I will go over a pretty reasonable set of tools and patterns to very common problems. And hopefully, this would be valuable for some people.

    Stay tuned for for next part, where we'll talk about some choices around persistence, and serving and storing the data.