Auto Response for Testing Node.js SaaS Based REST API

I have been enjoying the use of Node.js.  I started by first creating the backend which is a node.js server which has REST end-points that my client will call.  In this case the client is a mobile application to receive data on tweets.  That isn’t necessarily important for this short post.

I was at a coffee shop recently and really didn’t want to ping my node.js test server each time I was working on the client.  I used Fiddler to create an auto-response that would usually come back from the node.js server.

Fiddler Auto Response

1. In the picture you can see that I have right-clicked on the request to save the response.

2. Click on the AutoResponder tab.

3. Then insert your the response url you want to auto repsond to.  Add in the file location to the response you saved from the last step.

4. I clicked the check box above which allows other traffic to pass through so that I could still go to other websites.

Sencha Touch 2.2.1 Update from 2.0.x

Few things I encountered as I was updating to the latest version.  Hopefully this can help others:

Error: com.sencha.exceptions.ExProcess: compass process exited with non-zero code:1

  • Make *completely* sure you have ruby 1.9.3 installed and not 2.0.0.  If on windows use this installer.
  • Install compass from gem (ie. gem install compass).  Don’t install from node.js package manager.

Error: File not found or cannot be read: sass/stylesheets/fonts/pictos/pictos-web.woff

Error: Sass::SyntaxError on line [“190”] of C: Undefined variable: “$theme-name”.

  • If you re-used the “&.popular” selector you need to remove it now.

Ubuntu Upgrade 8.10 to 10.04 – Brain Dump

Update Ubuntu from 8.10 to 10.04

Had IT clone the production VM to have test VM for trying the Ubuntu upgrade. Had trouble getting cloned VM connected to the network.

Upgrade 8.10 to 9.04

Instructions from EOLUpgrades and EOLUpgrades Intrepid pages.

*Be sure to have all the 8.10 updates before upgrading

sudo apt-get update
sudo apt-get upgrade
  • From version 6.06 and up you will need to install the update-manager and update-manager-core packages.
sudo aptitude install update-manager-core update-manager
  • Also make sure that the update manager is correctly configured to upgrade any release:
sudo perl -pi -e 's/^Prompt=.*/Prompt=normal/' /etc/update-manager/release-upgrades
  • Kernel update
sudo aptitude install linux-image-virtual linux-headers-virtual
  • Update the package list and upgrade all the installed packages
sudo aptitude update && sudo aptitude safe-upgrade
  • Perform the release upgrade.
sudo do-release-upgrade

As upgrade runs…

  • Important note for NIS/YP users â
      â                                                                                          â
      â To use MySQL, the following entries for users and groups should be added to the system:  â
      â                                                                                          â
      â  /etc/passwd   : mysql:x:100:101:MySQL Server:/var/lib/mysql:/bin/false                  â
      â  /etc/group    : mysql:x:101:                                                            â
      â                                                                                          â
      â                                                                                          â
      â You should also check the permissions and the owner of the /var/lib/mysql directory:     â
      â                                                                                          â
      â  /var/lib/mysql: drwxr-xr-x   mysql    mysql                                             â

When the upgrade runs it will prompt that certain files have changed. Accept the new version and manually make changes to it :

  • /etc/security/limits.conf
-* soft nofile 5536
-* hard nofile 5536
-
-
-#cainteg                 hard    nofile         5000
  • /etc/exports
  â --- /etc/exports 2010-12-30 14:09:04.000000000 -0600                                   â
  â +++ /usr/share/nfs-kernel-server/conffiles/etc.exports 2008-12-04 16:43:12.000000000   â
  â -0600                                                                                  â
  â @@ -2,11 +2,9 @@                                                                       â
  â  # to NFS clients. See exports(5).                                                     â
  â  #                                                                                     â
  â  # Example for NFSv2 and NFSv3:                                                        â
  â -# /srv/homes hostname1(rw,sync) hostname2(ro,sync)                                    â
  â +# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)  â
  â  #                                                                                     â
  â  # Example for NFSv4:                                                                  â
  â -# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt)                                        â
  â -# /srv/nfs4/homes gss/krb5i(rw,sync)                                                  â
  â +# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)                       â
  â +# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)                                 â
  â  #                                                                                     â
  â -/mnt/disk1 *(rw,sync,no_root_squash)                                                  â
  â -/srv/git *(rw,sync,no_root_squash)
  • /etc/apache2/ports.conf
--- /etc/apache2/ports.conf     2009-03-16 16:59:14.000000000 -0500
+++ /etc/apache2/ports.conf.dpkg-new    2010-08-16 12:44:05.000000000 -0500
@@ -1,11 +1,12 @@
 # If you just change the port or add more ports here, you will likely also
 # have to change the VirtualHost statement in
 # /etc/apache2/sites-enabled/000-default
+# This is also true if you have upgraded from before 2.2.9-3 (i.e. from
+# Debian etch). See /usr/share/doc/apache2.2-common/NEWS.Debian.gz and
+# README.Debian.gz 

 NameVirtualHost *:80
 Listen 80
-Listen 10200
-Listen 3000
  • /etc/apache2/sites-available/default
  • Prompt about phpmyadmin. Auto configure db? said ‘OK’
  • /etc/phppgadmin/apache.conf
--- /etc/phppgadmin/apache.conf 2009-03-13 08:46:40.000000000 -0500
+++ /etc/phppgadmin/apache.conf.dpkg-new        2009-02-09 08:23:37.000000000 -0600
@@ -10,12 +10,11 @@
order deny,allow
deny from all
allow from 127.0.0.0/255.0.0.0 ::1/128
-allow from all
+# allow from all

<IfModule mod_php5.c>
  php_flag magic_quotes_gpc Off
  php_flag track_vars On
-  php_flag register_globals On
  php_value include_path .
</IfModule>
  • Remove obsolete packages?
2 packages are going to be removed.
Continue [yN]  Details [d]d
Continue [yN]  Details [d]y

After upgrade completes…

  • When the upgrade completes it will prompt that a restart is required. After the restart, run lsb_release -a to check the new version of Ubuntu.
lsb_release -a
  • Cleanup to have more disk space
sudo apt-get clean
  • Take a snapshot, then upgrade from 9.04 to 9.10
sudo shutdown now

Upgrade 9.04 to 9.10

Instructions from EOLUpgrades Jaunty page.

  • sources.list should have jaunty in it
vi /etc/apt/sources.list
  • Update the package list and upgrade all the installed packages
sudo aptitude update && sudo aptitude safe-upgrade
  • Perform the release upgrade.
sudo do-release-upgrade

As Upgrade Runs

  • SSH
If you continue, a additional ssh daemon will be started at port '9004'.
Do you want to continue?
Continue [yN] y
  • Prompted :
  ââââââââââââââââââââââââââââââââ⤠Configuring mysql-server-5.1 ââââââââââââââââââââââââââââââââââ
   â                                                                                               â
   â Important note for NIS/YP users                                                               â
   â                                                                                               â
   â Using MySQL under NIS/YP requires a mysql user account to be added on the local system with:  â
   â                                                                                               â
   â  adduser --system --group --home /var/lib/mysql mysql                                         â
   â                                                                                               â
   â                                                                                               â
   â You should also check the permissions and ownership of the /var/lib/mysql directory:          â
   â                                                                                               â
   â  /var/lib/mysql: drwxr-xr-x   mysql    mysql                                                  â
   â                                                                                               â
   â                                            <Ok>                                               â
   â                                                                                               â
   âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
  • Override local changes to /etc/pam.d/common-*?
sudo cp /etc/pam.d/common-* .
  • libpam-runtime files modified. Made copies and accepted new versions – may need to diff those to find what changes we need to copy/paste back in.
  • Prompted
  âââââââââââââââââââââââââââââââââ⤠Configuring postgresql-common âââââââââââââââââââââââââââââââââââ
  â                                                                                                  â
  â Obsolete major version 8.3                                                                       â
  â                                                                                                  â
  â The PostgreSQL version 8.3 is obsolete, but the server or client packages are still installed.   â
  â Please install the latest packages (postgresql-8.4 and postgresql-client-8.4) and upgrade the    â
  â existing  clusters with pg_upgradecluster (see manpage).                                         â
  â                                                                                                  â
  â Please be aware that the installation of postgresql-8.4 will automatically create a default      â
  â cluster 8.4/main. If you want to upgrade the 8.3/main cluster, you need to remove the already    â
  â existing 8.4 cluster (pg_dropcluster --stop 8.4 main, see manpage for details).                  â
  â                                                                                                  â
  â The old server and client packages are no longer supported. After the existing clusters are      â
  â upgraded, the postgresql-8.3 and postgresql-client-8.3 packages should be removed.               â
  â                                                                                                  â
  â Please see /usr/share/doc/postgresql-common/README.Debian.gz for details.                        â
  â                                                                                                  â
  â                                              <Ok>                                                â
  â                                                                                                  â
  ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
  • /etc/apache2/apache2.conf changed – loglevel to warn instead of info, changed logging format. Kept new version.

After upgrade completes…

  • When the upgrade completes it will prompt that a restart is required. After the restart, run lsb_release -a to check the new version of Ubuntu.
lsb_release -a
  • Cleanup to have more disk space
sudo apt-get clean
  • Take a snapshot, then upgrade from 9.10 to 10.04
sudo shutdown now

Upgrade from 9.10 to 10.04

Instructions from Lucid page.

  • Install update-manager-core if it is not already installed:
sudo apt-get install update-manager-core
  • edit /etc/update-manager/release-upgrades and set Prompt=normal (was already normal)
  • Launch the upgrade tool:
sudo do-release-upgrade
  • Follow the on-screen instructions.

As Upgrade Runs…

  • /etc/dbconfig – accepted new version
  • /etc/auto.master
              â -# $Id: auto.master,v 1.4 2005/01/04 14:36:54 raven Exp $                         â
              â -#                                                                                â
              â  # Sample auto.master file                                                        â
              â  # This is an automounter map and it has the following format                     â
              â  # key [ -mount-options-separated-by-comma ] location                             â
              â  # For details of the format look at autofs(5).                                   â
              â -#/misc /etc/auto.misc --timeout=60                                               â
              â -#/smb /etc/auto.smb                                                              â
              â +#                                                                                â
              â  #/misc /etc/auto.misc                                                            â
              â -/net /etc/auto.net                                                               â
              â +#                                                                                â
              â +# NOTE: mounts done from a hosts map will be mounted with the                    â
              â +# "nosuid" and "nodev" options unless the "suid" and "dev"                       â
              â +# options are explicitly given.                                                  â
              â +#                                                                                â
              â +#/net -hosts                                                                     â
              â +#                                                                                â
              â +# Include central master map if it can be found using                            â
              â +# nsswitch sources.                                                              â
              â +#                                                                                â
              â +# Note that if there are entries for /net or /misc (as                           â
              â +# above) in the included master map any keys that are the                        â
              â +# same will not be seen as the first read key seen takes                         â
              â +# precedence.                                                                    â
              â +#                                                                                â
              â ++auto.master
  • /etc/gitweb.conf
--- /etc/gitweb.conf    2009-11-23 09:23:40.000000000 -0600
+++ /etc/gitweb.conf.dpkg-new   2010-12-17 18:57:49.000000000 -0600
@@ -1,16 +1,11 @@
-#Ryan's added
-$my_uri = "http://ah-git/gitweb";
-$site_name = "ah-git/gitweb";
-$project_list_description_width = 40;
-
 # path to git projects (<project>.git)
-$projectroot = "/srv/git";
+$projectroot = "/var/cache/git";

 # directory to use for temp files
 $git_temp = "/tmp";

 # target of the home link on top of all pages
-$home_link = $my_uri || "/";
+#$home_link = $my_uri || "/";

 # html text to include at home page
 $home_text = "indextext.html";
@@ -19,10 +14,10 @@
 $projects_list = $projectroot;

 # stylesheet to use
-$stylesheet = "gitweb.css";
+$stylesheet = "/gitweb/gitweb.css";

 # logo to use
-$logo = "/git-logo.png";
+$logo = "/gitweb/git-logo.png";

 # the 'favicon'
-$favicon = "/git-favicon.png";
+$favicon = "/gitweb/git-favicon.png";
  • RPM db cleaned up – chose OK
 ââââââââââââââââââââââââââââââââââââââââââââââ⤠Configuring rpm âââââââââââââââââââââââââââââââââââââââââââââââ
 â                                                                                                             â
 â Package information database for rpm cleaned up                                                             â
 â                                                                                                             â
 â The database which rpm keeps about installed packages is not usable with the new version.                   â
 â                                                                                                             â
 â The old database has been moved to /var/backups. Please read /usr/share/doc/rpm/README.Debian for details.  â

After upgrade completes…

  • When the upgrade completes it will prompt that a restart is required. After the restart, run lsb_release -a to check the new version of Ubuntu.
lsb_release -a
  • Cleanup to have more disk space
sudo apt-get clean
  • Take a snapshot

Changing Hostname in Ubuntu

To permanently change your host name make sure you hit all the spots.  You can start by looking at the following:

1
sudo hostname NEW_HOST

You will first want to edit your hostname file:

1
sudo vi /etc/hostname

Then run:

1
/etc/init.d/hostname.sh start

After you have done that then change your hosts file:

1
sudo vi /etc/hosts

If you are running this on VMWare you will need to delete this file

There is a file /etc/udev/rules.d/<some number>_persistent-net.rules, where udev caches, which hardware device should get named eth0. Here it looks like this:

1
2
\# PCI device 14xx:16xx (tg3)
ACTION=="add", SUBSYSTEM=="net", DRIVERS=="?*", SYSFS\{address}=="00:11:xx:xx:xx:xx", NAME="eth0"

Make sure that you have configured to use dhcp instead of static ip for your machine. Otherwise it will grab the same ip address of the other machine already on your network.

passwd: Authentication token manipulation error

There are many reasons why you might get this error but a simple one (not to overlook) is that you have your username on NIS! In my case, I have local and NIS users and forgot that this user was hooked up to NIS.

Instead of using passwd you should be using:

1
yppasswd

ConverterNotFound Using Spring Roo

Using Roo has been really awesome! I would recommend it to anyone starting a web project for the first time that really wanted to use Spring and Java.

Just ran into this error tonight and thought I would blog a little bit about it in case others run into the same error… *plus* I wanted to show a bit of this new Google Guava project!

1
2
3
4
org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from 'java.util.Se
t'
to 'java.lang.String'
        at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:181)
        at org.springframework.expression.spel.support.StandardTypeConverter.convertValue(StandardTypeConverter.java:66)

This meant that we need a converter for the show view since the default create from Roo does not take care of the ONE_TO_MANY relationship. Below is the code that was needed in my case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/**
 * A central place to register application Converters and Formatters.
 */

@RooConversionService
public class ApplicationConversionServiceFactoryBean extends FormattingConversionServiceFactoryBean {

    Converter<Set<Property>, String> getPropertySetConverter() {
        return new Converter<Set<Property>, String>() {
            public String convert(Set<Property> properties) {
                return Joiner.on(",").join(properties.toArray());      // 1
            }
        };
    }
  @Override
  protected void installFormatters(FormatterRegistry registry) {
    super.installFormatters(registry);
    registry.addConverter(getPropertySetConverter());
  }
 
}

Notice the line:

1
return Joiner.on(",").join(properties.toArray());      // 1

This is using the Google Guava project to create a String instead of using the old school StringBuilder approach.

Mysqld Fails to Start

When I ran mysqld I would get:

> 090127 10:00:30 InnoDB: Operating system error number 13 in a file
> operation.
> InnoDB: The error means mysqld does not have the access rights to
> InnoDB: the directory.
> InnoDB: File name ./ibdata1
> InnoDB: File operation call: ‘open’.
> InnoDB: Cannot continue operation.

Finally I looked in /etc/mysql/my.cnf and figured out that my bind-address needed to be updated. So took about an hour to figure that one out. It would be best if I could just leave it to localhost for the bind_address but I can not do that right now.

For others where it is not the bind_address it could be number of other things. I started reading through this thread: http://lists.mysql.com/mysql/216042.  It was very helpful to think through the scenarios.

Gerrit Workflow

The workflow defines a process for using with Google’s Gerrit project. It is excellent in a number of ways! If you are using Git at your company and want to have a nice code review process as well as a way to verify and have a few more checks and balances, this is an amazing tool that Google has provided open source. I did make a few customizations to the tool in order to better suite our needs within the company and maybe I will share those once I get some more free time.

The idea with the following workflow is to show you a developer process from beginning to the end of a feature using Gerrit.  We are not using Repo which is a Google developed tool to work with Git and Gerrit.  We are using straight out of the box Git with Gerrit.  In our company we have a few proprietary development build processes but I try to call those out in the workflow so that you can tell how it could be adapted to your company or open source project.

Workflow

  • Implement feature
  • 1
    2
     //Create topic branch off of master
    git checkout -b topic origin/master
  • Commit changes. Commit as many times as you want to the topic branch.
  • Get latest code frequently and especially before you push.
  • 1
    git pull
  • Fix Merge Conflicts
  • 1
    2
    3
     git mergetool
    //'''Do *NOT*''' commit after you do the merge
    git rebase --continue
  • Compile the code
  • 1
    2
    3
    4
    //This is an internal shortcut for us to do a full compile just like the integration hudson builds
    //You really want to be able to replicate exactly what the integration machines are doing and
    //make it as easy as possible.
    projbuild
  • Validate the code (This means running your unit tests, integration tests, a subset of all the selenium tests, etc.)
  • 1
    projbuild dev_check
  • Prepare a single commit for code review by squashing your commits
  • Create another local branch
  • 1
    git checkout -b topicForCodeReview origin/master
  • Merge squash your topic branch into your topicForCodeReview branch
  • 1
    git merge --squash topic
  • The squash leaves the changes in your topicForCodeReview branch in the staged state. You’ll need to fire up git gui and commit them to that branch.
  • Push for code review
  • 1
    git push for-code-review
  • Fix any merge issues using git mergetool if necessary.
  • Note: If you choose not to squash before pushing for code review, every commit will get a separate code review!! This is not recommended.

Setup

You will need to setup the git config a certain way in order to be able to pull off this workflow.  Pretty trivial changes though, so don’t worry to much yet 🙂

  • Set rebase to always
  • 1
    2
    [core]
      rebase = always
  • Set up the shortcut for “for-code-review”
  • 1
    2
    3
    [remote "for-code-review"]
      url = <main git uploading repository location>
      push = HEAD:refs/for/master

Apache Log Maintenance

Useful for those managing apache sites.  This will set the rotating logs so that it is easier to review issues and problems when they occur.  This is also much needed so that one log file does not get to large and unmanageable.

On Linux:

CustomLog “|bin/rotatelogs /var/logs/logfile 86400” common

On Windows:

CustomLog “|D:/viewstore/snapshot/ralberts/apache/bin/rotatelogs.exe D:/viewstore/snapshot/ralberts/apache/logs/access.%Y-%m-%d-%H_%M_%S.log 60” common

This information can also be found at apaches website.