Wednesday, July 16, 2008

Switchable Grails DataSource

As part of my day job I needed to write an administration app that could be easily pointed at multiple environments. To achieve this I used a technique I found a while ago based on Spring's AbstractRoutingDataSource. It basically works by using a facade to switch the connection between one to many child data sources. First off SwitchableDataSource.groovy
package com.leebutts 
import org.springframework.jdbc.datasource
                 .lookup.AbstractRoutingDataSource 
import com.leebutts.Environment 
import org.springframework.context.ApplicationContextAware 
import org.springframework.context.ApplicationContext 
import javax.sql.DataSource 
import org.springframework.jdbc.datasource.DriverManagerDataSource 

class SwitchableDataSource extends AbstractRoutingDataSource 
    implements ApplicationContextAware { 
    def applicationContext 
    
    public void setApplicationContext(ApplicationContext 
                                           applicationContext) { 
        this.applicationContext = applicationContext 
    } 

    protected DataSource determineTargetDataSource() { 
        DriverManagerDataSource ds = 
                        super.determineTargetDataSource(); 
        def env = EnvironmentHolder.getEnvironment() 
        if (env && env.passwordRequired && ds) { 
            ds.setPassword(env.password) 
        } 
        return ds 
    } 

    protected Object determineCurrentLookupKey() { 
        def env = EnvironmentHolder.getEnvironment() 
        return env?.id ?: Environment.list()[0]?.id 
    } 
}
SwitchableDataSource is the facade that delegates to the list of standard DriverManager data sources. They are defined in grails-app/conf/spring/resources.groovy using the environment settings from the Environment class. Here's my spring config in resources.groovy:
import com.leebutts.SwitchableDataSource 
import com.leebutts.Environment 
import org.springframework.jdbc.datasource.DriverManagerDataSource 

beans = { 
    parentDataSource(DriverManagerDataSource) {
        bean -> bean.'abstract' = true; 
        driverClassName = 'com.mysql.jdbc.Driver' 
        username = "root" 
    } 
    
    Environment.list().each {env -> 
        "${env.prefix}DataSource"(DriverManagerDataSource) {bean -> 
            bean.parent = parentDataSource 
            bean.scope = 'prototype' 
            def port = env.port ?: 3306 
            url = "jdbc:mysql://${env.host}:${port}/switchingDemo" 
            if (env.user) { 
                username = env.user 
            } 
            if (env.password) { 
                password = env.password 
            } 
        } 
    } 

    def dataSources = [:] 
    Environment.list().each {env -> 
        dataSources[env.id] = ref(env.prefix + 'DataSource') 
    } 

    dataSource(SwitchableDataSource) { 
        targetDataSources = dataSources 
    } 
}
Environment currently uses a static list to hold the environment config. This could be done better via a reloadable properties file or by adding a view/controller to modify the environment settings on the fly. Environment.groovy:
package com.leebutts

class Environment {

    static environments = []

    static {
        environments << [id: 1, name: 'local', prefix: 'local', 
                          host: 'localhost']
        environments << [id: 2, name: 'UAT', prefix: 'uat',
                            host: 'uat.leebutts.com']
        environments << [id: 3, name: 'Testing', prefix: 'testing',
                         host: 'testing.leebutts.com'] 
        environments << [id: 4, name: 'Beta', prefix: 'beta',
                         host: 'beta.leebutts.com', 
                         passwordRequired: true] 
        environments << [id: 5, name: 'Prod', prefix: 'prod', 
                         host: 'db.leebutts.com', user:'grails', 
                         port: 13306, 
                         passwordRequired: true] 

        //unique id check 
        environments.each {env ->
            assert environments
                .findAll {it.id == env.id}.size() == 1}
    }

    static list() {
        return environments
    }
}
SwitchableDataSource needs a way to determine which environment the current request wishes to use. It does this via a ThreadLocal holder EnvironmentHolder.groovy:
package com.leebutts

class EnvironmentHolder {

    private static final ThreadLocal contextHolder
                                = new ThreadLocal();

    static void setEnvironment(Map environment) {
        contextHolder.set(environment);
    }

    static getEnvironment() {
        return contextHolder.get();
    }

    static void clear() {
        contextHolder.remove();
    }
}
Now that the infrastructure is in place, the next step is to add a controller to allow users to select the environment they wish to use and a filter to set a default environment if one has not yet been chosen. The controller is called via ajax (in my application) but could be used as a standard controller just as easily. It looks up the Environment based on an ID and then tests the connection to make sure the password supplied is valid (if a password is required as specified in the environment config). If a password is being used it takes a copy of the environment config map, adds the password and stores it in the session so that the user doesn't have to re-enter the password on every screen and so that only this user has access to the password. EnvironmentController.groovy:
import com.leebutts.Environment
import com.leebutts.EnvironmentHolder
import javax.servlet.http.HttpServletResponse
import org.codehaus.groovy.grails.commons.ApplicationAttributes
import org.codehaus.groovy.grails.web.context.ServletContextHolder

class EnvironmentController {

    def change = {
        if (params.environment) {
            def env = Environment.list()
                     .find {it.id == new Integer(params.environment)}
            if (env) {
                if (env.passwordRequired) {
                    if (params.password) {
                        //take a copy and add a pword
                        env = addPasswordToEnvCopy(params, env)
                    } else {
                        render 'PASSWORD REQUIRED'
                        response.setStatus(
                          HttpServletResponse.SC_UNAUTHORIZED)
                        return
                    }
                }

                //test connection
                def oldEnv = EnvironmentHolder.getEnvironment()
                EnvironmentHolder.setEnvironment env
                def ds = getDataSourceForEnv(env)
                try {
                    def con = ds.getConnection()
                    session.environment = env
                    render 'Environment change complete.'
                } catch (e) {
                    EnvironmentHolder.setEnvironment oldEnv
                    render 'Unable to connect to database: '
                              + e.message
                    response.setStatus(
                       HttpServletResponse.SC_UNAUTHORIZED)
                    return
                }
            } else {
                render 'No such environment'
                response.setStatus(
                        HttpServletResponse.SC_BAD_REQUEST)
            }
        } else {
            render 'Missing parameter environment'
            response.setStatus(HttpServletResponse.SC_BAD_REQUEST)
        }
    }

    private def getDataSourceForEnv(env) {
        def servletContext = ServletContextHolder.servletContext
        def ctx = servletContext
                    .getAttribute(
                         ApplicationAttributes.APPLICATION_CONTEXT)
        return ctx.dataSource
    }

    private Map addPasswordToEnvCopy(Map params, env) {
        def myEnv = [:]
        env.each {key, val ->
            myEnv[key] = val
        }
        myEnv.password = params.password
        return myEnv
    }
}
As mentioned, there is also a simple filter for defaulting the environment to the first one in the list if one has not been selected and storing it in the ThreadLocal holder. Filters.groovy:
import com.leebutts.EnvironmentHolder
import com.leebutts.Environment

class Filters {
    def filters = {
        all(uri: '/**') {
            before = {
                if (!session.environment) {
                    session.environment = Environment.list()[0]
                }
                EnvironmentHolder.setEnvironment(session.environment)
            }
        }
    }
}
The final step is to add an environment selection form to your layout so that users can choose their environment. views/layouts/main.gsp:
<html>
<head>
    <title>Administration System</title>
    <link rel="stylesheet"
            href="${createLinkTo(dir: 'css', file: 'main.css')}"/>
    <g:layoutHead/>
    <g:javascript library="application"/>
    <g:javascript library="prototype"/>
    <script type="text/javascript">
        function refresh()
        {
            window.location.reload(false);
        }

        function loading() {
            document.getElementById('spinner').style.display = 'inline';
            document.getElementById('error').style.display = 'none';
        }
      
        function showError(e) {
            var errorDiv = document.getElementById('error')
            errorDiv.innerHTML = '<ul><li>'
                     + e.responseText + '</li></ul>';
            errorDiv.style.display = 'block';
        }
    </script>
</head>
<body>
<div class="logo">
    <div style="margin-left:10px;">
        <h1>Current Environment: 
               ${session.environment?.name ?: 'None'}</h1>
        <form action="">
            <g:select name="environment" 
                   from="${com.leebutts.Environment.list()}" 
                      optionKey="id" optionValue="name" 
                       value="${session.environment?.id}"/>
            <g:passwordField name="password"/>
            <g:submitToRemote value="Select" controller="environment" 
             action="change" update="currentEnv"
             onLoading="loading();"
             onComplete
               ="document.getElementById('spinner').style.display='none';"
             onFailure="showError(e)"
             onSuccess="refresh()"/> <br/>
            <div class="errors" id="error" 
              style="display:none;width:500px;">

            </div>
            <img id="spinner" style="display:none;"
              src="${createLinkTo(dir: 'images', file: 'spinner.gif')}"
              alt="Spinner"/>
        </form>
    </div>
</div>
<g:layoutBody/>
</body>
</html>
The finished screen looks something like this:
Photobucket - Video and Image Hosting

Photobucket - Video and Image Hosting

Photobucket - Video and Image Hosting

Photobucket - Video and Image Hosting

As this is an internal admin application it has not been tested as thoroughly as standard prod code so please do so before using it in a "live" application. Please post any corrections/bug fixes.

7 comments:

Shawn Hartsock said...

wow, that's nice. Thanks for posting that!

bruce said...

Lee

Thanks, a switchable datasource is just what I am after. I seemed to have cocked something up however, as I can never get Grails to use the chosen environment - it will only ever use the one specified in DataSource.groovy. If I don't specify one in DataSource.groovy I get a startup error on not being able to find any datasource.

Am I barking up the wrong tree or just barking?

Any tips you have regards DataSource.groovy would be greatly appreciated.

Regards

Bruce

Lee said...

Hi Bruce,

the changes made to resources.groovy should be overriding any settings in DataSource.groovy. I think I have my DataSource.groovy in the default HSQLDB memory database state.

Check that the dataSource bean doesn't have any typos (it must have an upper case S in it's name)

cheers

Lee

Bruce said...

Thanks Lee. It is now working fine. I had Resources.groovy instead of resources.groovy and had Filters.groovy under src/groovy instead of under grails-app/conf. As you can see, I am still finding my way around Grails. The ability to have switchable datasources by client is critical for my application (I would imagine for a lot of others as well) so I am very grateful for your work.

Regards

Bruce

malpropio said...

I would like to know if its possible to use this technique but instead have the user specify on the fly the datasource and not have them statically set up in the environment class.

Lee Butts said...

Hmmmm, you should be able to add new ones on the fly. You would need to dynamically add a new Spring datasource and then update the list of environments.

You would also need to update the targetDatasources list inside the SwitchableDataSource bean.

harish said...

Excellent. Precisely what I was looking for. Thanks for sharing.

-Harish