Google Web Toolkit on Google App Engine: Join in a performance test!

There's an interesting experiment using the Google Web Toolkit and Google App Engine posted here. I am very curious to see what sort of performance can be had when the GWT is paired with App Engine, with no explicit, special optimizations.

If you are free and want to join in the fun, just hit http://qwerty.codathlon.com on May 23, 2008, 16:00, Paris-time. That's 23:00 here in Sapporo, Japan.

I am so in.

Japan's IT state of the nation: bleak outlook

Good thread on slashdot regarding outsourcing to China.

Here in Sapporo, the local municipal government has a deal with IT companies based in Shenyang. Having personally been involved in project management capacity with a company in Shenyang recently, as well as having worked in the past with support and development teams based in mainland China, I am rather doubtful of the present Sapporo-Shenyang partnership. More so since there are no more direct flights between Sapporo and Shenyang.

I think that the odds are stacked against you in terms of meeting a project deadline on time and with high-quality, largely due to differences in cultural expectations and the lack of a good telecommunications infrastructure in Shenyang. But even more than that, afaik, there is a sorry lack of experience and understanding about such globalized, international endeavors here in Sapporo. It really is disconcerting to see the hit-or-miss approach being taken by some companies.

I suppose that this may have a lot to do with the dwindling numbers of university graduates with technology degrees. I do not think, however, that a strategy of retaining only project managers and outsourcing the bulk of work will succeed in the end, both for the enterprise and for the local and national economies.

RSpec and stubbing ssl_required

Here's a quick rspec stubbing tip for those of you using RSpec and Spec::Rails with a need to stub out the controller behavior for ssl_required. I wrote this tip with mocha in mind, but you can easily apply this to whatever mocking/stubbing framework you may be using.

First, let's take a look at the source for the SslRequirement module, paying attention to the the before_filter bit:

module SslRequirement
  def self.included(controller)
    controller.extend(ClassMethods)
    controller.before_filter(:ensure_proper_protocol)
  end

  ...

  private
  def ensure_proper_protocol
    return true if ssl_allowed?

    if ssl_required? && !request.ssl?
      redirect_to "https://" + request.host + request.request_uri
      flash.keep
      return false
    elsif request.ssl? && !ssl_required?
      redirect_to "http://" + request.host + request.request_uri
      flash.keep
      return false
    end
  end
end

You can see that it is the private method ensure_proper_protocol that is being declared as a filter for the controller. Since a controller that declares ssl_required will automatically inherit the methods listed in SslRequirement, you can stub the SSL-checking behavior in mocha-style for your controller's behavior-driven rspec test like so:

   controller.stub!(:ensure_proper_protocol).and_return(:true) 

Just declare the above stubbing in a setup block, and you are good to go!

Uninitialized Constant CGI::Session::ActiveRecordStore NameError

If you've found this post by keyword search, let me ask you: Are you using the Rails ActiveRecordStore along with the Ruby-GetText-Package ruby gem? And are you seeing an ugly, mysterious error that looks something this when starting Rails or even executing script/console:

[me@myserver]# mongrel_rails start
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with development environment...
/var/www/apps/argent/vendor/rails/actionpack/lib/../../activesupport/lib/active_support/dependencies.rb:478:in
  `const_missing': uninitialized constant CGI::Session::ActiveRecordStore (NameError)
  from /var/www/apps/argent/vendor/rails/actionpack/lib/action_controller/session_management.rb:24:in `const_get'
  from /var/www/apps/argent/vendor/rails/actionpack/lib/action_controller/session_management.rb:24:in `session_store='
  from /var/www/apps/argent/config/../vendor/rails/railties/lib/initializer.rb:328:in `send'
  from /var/www/apps/argent/config/../vendor/rails/railties/lib/initializer.rb:328:in `initialize_framework_settings'
  from /var/www/apps/argent/config/../vendor/rails/railties/lib/initializer.rb:327:in `each'
  from /var/www/apps/argent/config/../vendor/rails/railties/lib/initializer.rb:327:in `initialize_framework_settings'
  from /var/www/apps/argent/config/../vendor/rails/railties/lib/initializer.rb:324:in `each'
  from /var/www/apps/argent/config/../vendor/rails/railties/lib/initializer.rb:324:in `initialize_framework_settings'
   ... 15 levels...
  from /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'
  from /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281
  from /usr/local/bin/mongrel_rails:19:in `load'
  from /usr/local/bin/mongrel_rails:19

At the time of this post, I only saw three hits when Googling on "ruby", "rails", and "uninitialized constant CGI::Session::ActiveRecordStore (NameError)". 2 links to Japanese blog posts, the 3rd to a Rails forum in Polish. None helped me resolve the above problem.

But I was Googling in English.

When I tried changing tactics and Googling in Japanese instead, I found this link, and a hint that led me to the following solution...

There is some kind of require collision happening when trying to use both the ActiveRecordStore for session management and the gettext package for localization/internationalization. The answer is to put the require 'gettext/rails' declaration well after the default Rails environment bootstrapping. In my environment.rb, I've got my requires declarations set up like so:

# Bootstrap the Rails environment, frameworks, and default configuration
require File.join(File.dirname(__FILE__), 'boot')       # default
...

Rails::Initializer.run do |config|
# Use the database for sessions instead of the cookie-based default,
# which shouldn't be used to store highly confidential information
# (create the session table with 'rake db:sessions:create')
config.action_controller.session_store = :active_record_store
...
end

# After the Rails::Initializer.run block, just before end of file
require 'jcode'
require 'gettext/rails'

In doing so, you can avoid any namespace collisions between the default Rails requires and that for gettext/rails. Hope this saves you some time and headaches.

cron jobs for PostgreSQL users without using passwords

Ever needed to run some kind of cron job on a PostgreSQL database, but was stumped for not being able to specify a database user's password with the psql utility?

In using PostgreSQL with Ruby on Rails, I often have to set up regular cron jobs to clear out stale data, such as session data when using ActiveRecord as the session storage mechanism. Instead of coding up some object to react on a timer, you can simply use cron to do your bidding. But your regular cron job to connect to the PostgreSQL database needs a user, and that user needs to provide a password when connecting. Here's how you can set this up by creating a new UNIX/PostgreSQL user and configuring this user's database access in pg_hba.conf.

First, create a new UNIX account for your user. We will use the same username when creating a database user account as well. Why? Well, PostgresSQL allows connections not only over TCP/IP, but also over UNIX domain sockets. And, you can configure pg_hba.conf so that when a certain database user attempts to connect over a UNIX domain socket, PostgreSQL will check for an existing UNIX user account. If this UNIX user is the same as the user account attempting the database connection, then PostgreSQL will treat such access as trusted, hence no explicit password passing will be required.

For example, let's say that I want to clear out stale session data from my Rails sessions table. I first create a UNIX account for my database user.

# useradd victor

Don'f forget to set up a good password to victor. You may also want to revoke permissions from groups and others to access this user's home directory. For production environments, you want to be sure to assign victor to a group with limited UNIX user privileges, since we don't want people to misuse victor and mess with our database.

Next, we create a PostgreSQL database user account for victor, using the same username. We will create this database user with limited privilege on the sessions table, for security's sake. Assuming that the we are talking about the default Rails sessions table, we have:

# psql -U [database admin] [your_database]
psql@your_database> create user victor with password 'somepassword';
psql@your_database> grant select, delete on sessions to victor;

There! Now we have a PostgreSQL database user corresponding to our newly created UNIX user account.

Next, we add this line to our $PGDATA/pg_hba.conf file:

# TYPE  DATABASE            USER     CIDR-ADDRESS    METHOD
local   [your_database]     victor                   ident sameuser

This is how we allow the new user to connect to the PostgreSQL database using a UNIX domain socket connection. Logging into your UNIX machine as victor, you can now set up a simple cron job like so:

$ crontab -e
# crontab for victor
# Let's run a simple psql command as victor to delete stale rows in sessions table
# Execution time is
0 0 * * * /full/path/to/psql -U victor -c "delete from sessions where \
 updated_at < (now() - interval '6 hours')" [your_database] > /dev/null

So now we are good to go, with a UNIX/PostgreSQL user whose cron job will run daily at midnight, clearing out stale rows in the sessions table for our Rails ActiveRecord session management. And all without having to explicitly specify the password!

Apache to mongrel proxy permission denied on CentOS 5.1

Here's a brief tip for those of you about to set up Ruby on Rails, mongrel and Apache web server on CentOS 5.1: don't forget that you may have SELinux enforcement policies in place by default that prevent proxy balancers to connect correctly.

Now, you probably have proxy directives like the following in your httpd.conf file:

<Proxy balancer://mongrel_cluster>
  BalancerMember http://localhost:3000
  BalancerMember http://localhost:3001
  BalancerMember http://localhost:3002
</Proxy>

Firing up your mongrels and kicking up httpd will result in errors like this in your Apache errors logs:

[Thu Apr 24 23:24:20 2008] [error] (13)Permission denied: proxy: HTTP: attempt to connect to localhost:3000 (myserver) failed
[Thu Apr 24 23:24:20 2008] [error] ap_proxy_connect_backend disabling worker for (myserver)
[Thu Apr 24 23:24:21 2008] [error] proxy: BALANCER: (balancer://mongrel_cluster). All workers are in error state
[Thu Apr 24 23:24:21 2008] [error] proxy: BALANCER: (balancer://mongrel_cluster). All workers are in error state
[Thu Apr 24 23:24:21 2008] [error] proxy: BALANCER: (balancer://mongrel_cluster). All workers are in error state

Hm... alright, but what does /var/log/messages have to say?

Apr 24 23:24:19 myserver setroubleshoot:      SELinux is preventing the /usr/sbin/httpd from using potentially mislabeled files / (unlabeled_t).      For complete SELinux messages. run sealert -l 7e096a76-b66f-48f9-a30e-d736dbb6007d

Actually running that sealert command won't tell you any more beside the fact that SELinux is denying httpd access to potentially mislabeled files. Whatever that means. But perusing section 44.2.6. Enabling or Disabling Enforcement of the CentOS 5.1 deployment guide, there is enough there to resolve this obstruction.

Executing the sestatus command lets us see what SELinux is doing to protect our server.

[root@myserver]# sestatus -b | grep httpd
...
httpd_can_network_connect              off
...

Aha! This boolean property looks like it could be one that is denying the proxy connection from httpd to mongrel. So, let's enable this, shall we? The command to do this is setsebool. Reading the friendly man pages before running this command, we learn that setsebool will only set the current, running state of the boolean in question, and that if we desire to have this setting survive reboot, then we need to use the -P flag. Let us do exactly that:

[root@myserver]# setsebool -P httpd_can_network_connect=1

Now when you restart your mongrels and httpd, you will see that the proxy connections are good to go. And our configurations will live on, even after reboot. Sweet!

PostgreSQL 8.3.1 RPM's on CentOS 5.1

CentOS 5.1./RHEL5 comes with PostgreSQL 8.1.9, but if you are like me and want to enjoy the latest and greatest from the world's most advanced open source database, then you might want to try installing the PostgreSQL 8.3.1 RPM packages.

However, you will find that the postgresql-libs package has quite a few dependencies, not the least among them is apr-util, which is used by httpd. Deciding to go against the grain (and most common sense with respect to the entire premise of CentOS), I recently made a haphazard installation attempt whereby I blindly removed the postgresql-libs package with the rpm --nodeps option. My Apache web server refused to start, complaining about conflicts related to libpq.so.4 after I thought I successfully installed PostgreSQL 8.3.1. Luckily, this was all on VirtualBox, so recovery was not a problem.

Let me save you some time and headaches by outlining briefly the steps you can take to resolve this dependency issue by replacing the files upon which these dependencies lie by installing the compat-postgresql-libs-4, using the --replacefiles option.

First, go ahead and fetch the PostgreSQL RPMs for 8.3.1. And make sure you grab the aforementioned compat-postgresql-libs-4 RPM, too. I found all of the RPMs needed here at this archived repository.

Next, go ahead and start removing the 8.1.9 PostgreSQL packages with the rpm utility, saving the postgresql-libs package for last.

# rpm -e postgresql-contrib
# rpm -e postgresql-server
# rpm -e postgresql

Now when you attempt to remove the postgresql-libs package, you may see this:

# rpm -e postgresql-libs
error: Failed dependencies:
    libpq.so.4 is needed by (installed) apr-util-...
    libpq.so.4 is needed by (installed) exim-n...
    libpq.so.4 is needed by (installed) mod_auth_pgsql-...
    libpq.so.4 is needed by (installed) dovecot-...
    libpq.so.4 is needed by (installed) perl-DBD-Pg-...
　　...

The key here is libpq.so.4, which refuses to go quietly into the night when attempting to install PostgreSQL 8.3.1. So what are we to do? Well, let's take a closer look at what happens when we try to install the compat-postgresql-libs-4 RPM.

# rpm -i compat-postgresql-libs-4-1PGDG.rhel5.i686.rpm
     file /usr/lib/libpq.so.4 from install of compat-postgresql-libs-4-1PGDG.rhel5 conflicts with file from package postgresql-libs-8.1.9-1.el5
     file /usr/lib/libpq.so.4.1 from install of compat-postgresql-libs-4-1PGDG.rhel5 conflicts with file from package postgresql-libs-8.1.9-1.el5

Aha! Here we've found where the conflict lies. Let's work around this by replacing the existing libpq.so.4 from the original 8.1.9 with that from compat-postgresql-libs-4-1PGDG.rhel5.i686.rpm.

# rpm -iv --replacefiles compat-postgresql-libs-4-1PGDG.rhel5.i686.rpm

We are now home free! Continue by uninstalling the old postgresql-lib package, and you can then start the 8.3.1 package install. There should be no more trouble with dependencies for libpq.so.4, since it still exists but has been swapped out with that of compat-postgresql-libs-4-1PGDG.rhel5.i686.rpm. Good to go.