New Git Repository from an Existing Subdirectory

Posted on by

In this situation, I had been working in a git repository which had become to large and needed to be split into separate sub-modules. In this case, I really just needed to take out a subdirectory and make it a brand new repository. The great thing is that you can preserve all the history that had happened for that subdirectory if you decide that you want to keep it. There are a number of ways to do this process so this is just the way that I decided and liked the best.

Create new repository from existing subdirectory
First step I did was to make a clone of my existing repository (either by doing a regular copy/paste or a git clone).

Now by doing a filter-branch command we can use the existing git repository starting at the level of the subdirectory:

1
git filter-branch --subdirectory-filter foodir -- --all

After this is complete I needed to clean up and compact the repository:

1
2
3
git clean -d -f  //clean up untracked files
git gc --aggressive //clean up the repository packing
git prune

Done!

Remove history from existing full repository
This is optional if you want to re-write history from your existing repository. Some might just want to do a remove on the directory and then commit. I wanted to go back and re-write history to remove the commits to the subdirectory.

Then I did another clone of the repository which will clean from the history the subdirectory since I didn’t want that information to be in the repository anymore.

1
git filter-branch --index-filter 'git rm -rf --cached --ignore-unmatch dir' -- --all

Run a git gc again in this repository just to clean up everything.

Posted in GIT

JSON Output In Grails Controller

Posted on by

I wanted to be able to output an object with all the domain properties. This is obviously very easy in Grails:

1
MyObject as JSON

What if you wanted to be able to add to the MyObject? There are a bunch of ways that Grails provides in order to allow you to let you do this. Unfortunately, I didn’t know all of them at first so I had to do a bit of research.

Since I just wanted to add a simple property to render I added a transient value to my object:

1
static transients = ["link"]

Then I was able to set this value in the controller by iterating over my objects:

1
myObjectList.each() { it.href = createLink(action:'show', id:it.id).encodeAsHTML() };

Then I could just do the following:

1
render json as JSON

Bang! I have my extra attribute! Depending on what you want to do there might be a better way for your particular situation. As far as the code is concerned, this was a nice short change.

If more is needed I might need to start using the ObjectMarshaller. You can read about that and other ways to work with JSON by visiting the link below. They were probably the best resources I found out there at the moment.

Excellent blog:
Rending JSON Part 2: http://manbuildswebsite.com/2010/02/08/rendering-json-in-grails-part-2-plain-old-groovy-objects-and-domain-objects/

Rending JSON Part 3: http://manbuildswebsite.com/2010/02/15/rendering-json-in-grails-part-3-customise-your-json-with-object-marshallers/

Comcast Download/Upload Speed

Posted on by

I was considering doing a CLEAR plan which says it averages around 3-6 Mbps download speed. Not quite as fast as I would want. My guess is that the CLEAR plan does not truly ever reach 6 Mbps.

Think I will stick with Comcast for a while…
Comcast Speedtest

Selenium Localization

Posted on by

Not sure how other corporations are localizing Selenium. It has been an issue with ours. Since the creation of selenium we have not embedded localization into the infrastructure…oops, right?! The funny thing is that it took two of us to implement this simple solution in a couple hours. Once done we sent it off for review by the Automation Testing team. I wanted to let others take a look in case they were in the same situation that we were in.

One key is that we call into our Java server using a JSP would change depending upon how you get your localization entries for selenium.

How we localized Selenium:

•Added a class RBUtil that has methods overloaded method getValue(). As an example it is used in the following way:

1
2
3
4
5
String localizedValue = RBUtil.getValue(KEY_TO_LOCALIZATION)

//There are also signatures for the following:
RBUtil.getValue(key, default)
RBUtil.getValue(key, text insert array)

• The way this works is by calling the server with the resource bundle key. We are using the same Tomcat session as selenium to get the localized value.
• A key is that we are re-using the Tomcat session id so that we are not creating new sessions for each resource bundle (or localized string).
• We also cache the locale, username, etc. In fact, we will probably need to do more caching in the future but the main goal is that this will get us on our feet for having selenium localized.

Implementation Class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
package com.ptc.selenium.localization;

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.util.List;

import org.apache.log4j.Logger;

import sun.misc.BASE64Encoder;

import com.ptc.selenium.SeleniumLogger;
import com.ptc.selenium.SeleniumSession;
import com.ptc.selenium.SolventSelenium;
import com.ptc.selenium.SolventTestCase;

/**
 * Singleton class that is used to get the localization values from the RBInfo class file entries.
 *
 * This will retrieve values for selenium tests to be able to localize and attribute clicks to rbinfo files.
 *
 */
public class RBUtil {

    private static final Logger log = SeleniumLogger.getLogger(SolventTestCase.class.getName());

    private static RBUtil singleton;

    private final String baseURL;

    private final String username;

    private final String password;

    private final String cookie;

    private final String locale;

    private RBUtil() {
        SolventSelenium selenium = SeleniumSession.get();

        // Cache these values on the instance
        baseURL = selenium.getBaseURL();
        password = selenium.getPassword();
        username = selenium.getUsername();
        cookie = selenium.getCookieByName("JSESSIONID");
        locale = selenium.getEval("selenium.browserbot.getCurrentWindow().getLocale()");
    }

    public static RBUtil getInstance() {
        if (singleton == null) {
            singleton = new RBUtil();
        }
        return singleton;
    }

    /**
     * Will retrieve the resource bundle value given the key in the form of class.key. For example,
     * com.ptc.windchill.enterprise.folder.folderResource.PARTS_VIEW
     *
     * @param key
     *            In the form of CLASS.KEY. For example,
     *            "com.ptc.windchill.enterprise.folder.folderResource.PARTS_VIEW".
     * @return The value returned from the resource bundle lookup. If the resource could not be found empty string is
     *         returned
     */
    public static String getValue(String key) {
        return getValue(key, "");
    }

    /**
     * Will retrieve the resource bundle value given the key in the form of class.key.
     *
     * @param key
     *            In the form of CLASS.KEY. For example,
     *            "com.ptc.windchill.enterprise.folder.folderResource.PARTS_VIEW".
     * @param def
     *            The default value you want in the case that the locale could not be looked up.
     * @return The value returned from the resource bundle lookup.
     */
    public static String getValue(String key, String def) {
        String value = getInstance().getValueFromServer(key);
        return (value == null) ? def : value;
    }

    /**
     * Gets the localized string, unescapes the quot and puts the inserts in.
     *
     * @param key
     *            In the form of CLASS.KEY. For example,
     *            "com.ptc.windchill.enterprise.folder.folderResource.PARTS_VIEW".
     * @param inserts
     *            A list of inserts in order of the numbering in the rbinfo file.
     * @return The final string.
     */
    public static String getValue(String key, List<String> inserts) {
        String value = getInstance().getValueFromServer(key);

        for (int i = 0; i < inserts.size(); i++) {
            value = value.replace("{" + i + "}", inserts.get(i));
        }

        value = value.replaceAll("&quot;", """);

        return value;
    }

    private String getValueFromServer(String key) {
        InputStream ins = null;
        try {
            log.debug("Using Session Cookie = " + this.cookie);
            URL url = new URL(this.baseURL + "/netmarkets/jsp/util/getBundleKey.jsp?key=" + key);
            URLConnection conn = url.openConnection();

            conn.setRequestProperty("Cookie", "JSESSIONID=" + this.cookie);
            conn.setRequestProperty("Accept-Language", this.locale);
            conn.setRequestProperty("Authorization", "Basic " + encode(this.username + ":" + this.password));

            String value = null;
            try {
                ins = conn.getInputStream();
                BufferedReader reader = new BufferedReader(new InputStreamReader(ins));
                value = reader.readLine();
                log.debug("Fetched RbInfo Key=" + key + " Value=" + value);
            } finally {
                if (ins != null) {
                    ins.close();
                }
            }

            return value;
        } catch (Exception e) {
            throw new RuntimeException("Unable to reach server.", e);
        }
    }

    /**
     * Encode the string for use in the header.
     *
     * @param source
     *            The string you want to encode using base 64 encoding.
     * @return The encoded string.
     */
    public static String encode(String source) {
        BASE64Encoder enc = new sun.misc.BASE64Encoder();
        return (enc.encode(source.getBytes()));
    }

}

Client Side Git Hook

Posted on by

A client side Git hook that I wrote and are using at work. Thought others might have a similar need. Basically it will check the selenium file for the owners and see if our group is in that ownership. If the file is not in our group a message is displayed saying that the owner will need to get permissions and code review by the owners.

Of course you can use any language you want to execute your client side git hook. I thought shell scripting would just be easiest in my case since it wasn’t going to be that much code and anyone that has git will either be running in windows with cygwin and thus of bash or they will be in Linux.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
#!/bin/bash
#
# This hook is meant to do a pre-check to see if we have
# files outside of common components that need dac
# permissions
#
# Contact: Ryan Alberts
#
#############################################################

#Get the modified files
files_modified=`git diff --cached --name-only`

#echo "The present working directory is `pwd`"

#false or 0 if we do not need dac permissions
need_dac=0

#Commit message
commit_msg=`cat $1`

if [[ $commit_msg == *DAC* ]]
then
    exit 0
fi

#Go through the files in the submission
for file in ${files_modified[@]}
do
    accepted_path=0
   
    check=`grep -s wc-caIntegSprintTester $file`
   
    #Check if it is apart of our package
    if [[ $check == *wc-caIntegSprintTester* ]]
    then
        accepted_path=1
    fi

    if [ $accepted_path -eq 0 ]
    then
        echo "File is owned by another team, and may require DAC permissions!"
        echo "  File: $file"
        need_dac=1
    fi
   
done

#Exit Code For Commit Message Warning Or Not
if [ "$need_dac" -eq "1" ]
then
    echo ""
    echo ">>> Notice <<<"
    echo "Please contact the branch master and work on requesting DAC permissions for the files listed above."
    echo "You can avoid seeing this message by adding 'DAC' anywhere within the commit message."
    echo ""

    exit 1
fi

#Everything looks good - allow them to commit.
exit 0

Gerrit Workflow

Posted on by

The workflow defines a process for using with Google’s Gerrit project. It is excellent in a number of ways! If you are using Git at your company and want to have a nice code review process as well as a way to verify and have a few more checks and balances, this is an amazing tool that Google has provided open source. I did make a few customizations to the tool in order to better suite our needs within the company and maybe I will share those once I get some more free time.

The idea with the following workflow is to show you a developer process from beginning to the end of a feature using Gerrit.  We are not using Repo which is a Google developed tool to work with Git and Gerrit.  We are using straight out of the box Git with Gerrit.  In our company we have a few proprietary development build processes but I try to call those out in the workflow so that you can tell how it could be adapted to your company or open source project.

Workflow

  • Implement feature
    • 1
    2
     //Create topic branch off of master
    git checkout -b topic origin/master
  • Commit changes. Commit as many times as you want to the topic branch.
  • Get latest code frequently and especially before you push.
    • 1
    git pull
  • Fix Merge Conflicts
    • 1
    2
    3
     git mergetool
    //'''Do *NOT*''' commit after you do the merge
    git rebase --continue
  • Compile the code
    • 1
    2
    3
    4
    //This is an internal shortcut for us to do a full compile just like the integration hudson builds
    //You really want to be able to replicate exactly what the integration machines are doing and
    //make it as easy as possible.
    projbuild
  • Validate the code (This means running your unit tests, integration tests, a subset of all the selenium tests, etc.)
    • 1
    projbuild dev_check
  • Prepare a single commit for code review by squashing your commits
  • Create another local branch
    • 1
    git checkout -b topicForCodeReview origin/master
  • Merge squash your topic branch into your topicForCodeReview branch
    • 1
    git merge --squash topic
  • The squash leaves the changes in your topicForCodeReview branch in the staged state. You’ll need to fire up git gui and commit them to that branch.
  • Push for code review
    • 1
    git push for-code-review
  • Fix any merge issues using git mergetool if necessary.
  • Note: If you choose not to squash before pushing for code review, every commit will get a separate code review!! This is not recommended.

Setup

You will need to setup the git config a certain way in order to be able to pull off this workflow.  Pretty trivial changes though, so don’t worry to much yet 🙂

  • Set rebase to always
    • 1
    2
    [core]
      rebase = always
  • Set up the shortcut for “for-code-review”
    • 1
    2
    3
    [remote "for-code-review"]
      url = <main git uploading repository location>
      push = HEAD:refs/for/master

Testing Ext Objects Using JsUnit

Posted on by

We are using JsUnit to test our JavaScript code.  Most of the time we end up needing to stub out methods, save the method pointers and then put the original method back once done with the test.  This is very easy to do in JavaScript so why not!

An example test method usually looks like this:

1
2
3
4
5
6
7
8
9
10
11
function test_orderTabs() {
  var originalFunction = PTC.infoPage.TabSet.fooMethod;
  PTC.infoPage.TabSet.fooMethod = function() {
    //Do somthing else instead
  };
 
  //Do testing
 
  //Put back original function
  PTC.infoPage.TabSet.fooMethod = originalFunction;
}

Recently while doing some test driven development, I thought of extending our Ext component to make a Mock object.  This is really the same idea that we use in Java for JUnit testing.  The nice thing that happens when we do this in JavaScript is that we don’t need to worry about cleaning up function pointers!

1
2
3
4
5
6
7
8
9
PTC.infoPage.TabSetMock = Ext.extend(PTC.infoPage.TabSet, {
  fooMethod: function(){
  //Do Stuff
}
});
 
tabSetMock = new PTC.infoPage.TabSetMock({
  fooConfig: BarItems
});

Then later in the test method all I needed to do was something like the following:

1
2
3
function test_orderTabs() {
  var result = tabSetMock.fooMethod();
}

The benefit is that you don’t have to save function pointers. The mock was created in the context of the test method so once the test is over the memory of the mock is eased.

Apache Log Maintenance

Posted on by

Useful for those managing apache sites.  This will set the rotating logs so that it is easier to review issues and problems when they occur.  This is also much needed so that one log file does not get to large and unmanageable.

On Linux:

CustomLog “|bin/rotatelogs /var/logs/logfile 86400” common

On Windows:

CustomLog “|D:/viewstore/snapshot/ralberts/apache/bin/rotatelogs.exe D:/viewstore/snapshot/ralberts/apache/logs/access.%Y-%m-%d-%H_%M_%S.log 60” common

This information can also be found at apaches website.