In 2016, InnoGames started a new service department called the Core Department. One of the departments responsibilities is to provide centralized packages (payment, single-sign-on, general tools, etc.) for Unity-based games and an environment around to deliver and to also having the games collaborating with us on those packages easily.

As being one the developers starting the core department, we needed to think about the foundation for a system that is resolving centralized packages for games. What would be important for our games? What would they need at first? How can we provide a framework that enables developers embedded on the games to collaborate with us?

Our first task was to research existing solutions that we could adapt to the company needs. After this phase was completed, we came to the conclusion that there is not currently a solution which fit our needs on the market. Therefore, we came up with a vision of how the solution we wanted should look like.

As we already had an existing archiva (which is a repository management solution) we thought about solutions taking advantage of it. Gradle turned out to be a very good companion supporting us here. 

Dependency Resolver

The dependency resolver is a unity solution mainly based on gradle. Its purpose is to use a list of dependencies stored in a file, to resolve them from a central repository and to provide them to the Unity client. It is also capable of running optional post-resolve steps for every package.

The following flowchart shows how the dependency resolver works on a high-level perspective. A Unity user needs to click the Core Menu item for resolving dependencies after he cloned the repository, updated the pom file or switched the branch. The resolver parses the pom file and asks the Archiva repository for the dependencies (first level and transitive). This artifacts will be downloaded and stored in pre-defined folders inside of Unity. Each package can have a custom installer script that executes post-resolve tasks (for example: copying classes that derive from MonoBehaviour or ScriptableObject to a folder that is not excluded from the games repository). The game will build their implementations on top of the core packages.

Let’s resolve

Let’s start with the interesting part, the code. As written, we are executing the gradle part out of Unity with this command (shown below). I will go into detail of every task while we are looking at the code. The -u parameter prevents gradle searching for a settings file in parent folders. We ran into the issue that one of our games was also using gradle for other tasks and the resolver got stuck because he was trying to use the games gradle settings file in another folder.

../gradlew prepareUpdate unzipAll installPackage -u

Now we load a properties file and store the root folder defined there in a variable. The properties file contains some relative paths to folders that we need later on as well as the current version of the dependency resolver, this enables us to have the ability to decide later on if we actually need to update or not.

version = 0.5.7
sourcePomFolder = src/
destPomFolder = ../
pomName = pom.xml
 
# from root folder (InnoCore)
buildToolFolder = buildTool
buildToolFolderToRoot = ../
 
dependenciesResolverFolder = buildTool/dependenciesResolver
dependenciesResolverFolderToRoot = ../../
 
firstInstallFolder = buildTool/install/firstInstall
firstInstallFolderToRoot = ../../../
 
coreDependenciesFolder = CoreDependencies
coreDependenciesFolderToRoot = ../
 
tempUpdateToolFolder = tempUpdateToolFolder
tempUpdateToolFolderToRoot = ../
 
updateToolFolder = buildTool/install/update/updateTool
updateToolFolderToRoot = ../../../../
 
updateInstallerMiscFolder = buildTool/install/update/installer/misc
updateInstallerMiscFolderToRoot = ../../../../../
 
updateInstallerSrcName = installer
 
updateInstallerDestFolder = tempUpdateToolFolder
 
cleanupUpdateFolder = buildTool/install/cleanup
cleanupUpdateFolderToRoot = ../../../
 
gitIgnoreDest = ../InnoCore
gitIgnoreName = .gitignore

// load the properties file
def props = new Properties()
file("../builds.properties").withInputStream {
    stream -> props.load(stream)
}
 
def rootFolder = props.dependenciesResolverFolderToRoot

We are using two configurations. One for dependencies and one for the dependency resolver (called build tool in our script). When we or any of our game teams are working on any of the packages, every push to the Core repositories will generate a SNAPSHOT version (A SNAPSHOT version can always be overwritten and updated, a normal release version is read only) this will then be announced in one of our Slack channels to inform the game teams that the version is ready. Unfortunately gradle kept SNAPSHOTS in the local cache, so we needed to set the time for keeping version in the cache to 0 as seen on line 21.

Here is an example of how a pom file looks like. We have three entries in this pom file. The first is the Build Tool (it will update automatically if the version gets changed). The second is our portal package as a SNAPSHOT version. This means, that the code in the portal package can change on every resolve (and therefore also break release builds). The third entry is our network package as a release version. This package can not be changed anymore and is save to use.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <dependencies>
        <dependency>
            <groupId>InnoCore</groupId>
            <artifactId>CoreBuildTool</artifactId>
            <version>0.5.7</version>
            <type>zip</type>
        </dependency>
        <dependency>
            <groupId>com.innogames.core.frontend</groupId>
            <artifactId>core-portal-unity</artifactId>
            <version>0.4.0-SNAPSHOT</version>
            <type>zip</type>
        </dependency>
        <dependency>
            <groupId>com.innogames.core.frontend</groupId>
            <artifactId>core-network-unity</artifactId>
            <version>0.4.8</version>
            <type>zip</type>
        </dependency>
    </dependencies>
</project>

Here is how such a slack chat message of a new available SNAPSHOT version would look like. In our case the version number always contains the JIRA ticket number which is also the name of the branch. The snippet is there so the games can directly copy it to their pom file:

Gradle doesn’t allow you to override transitive (dependency of a first-level dependency) release versions with SNAPSHOT versions, which totally makes sense to not accidentally bring a project into an error state with unknown changes in a SNAPSHOT. During development and especially while working on one dependency deeper in the tree, you may want to override only this dependency with a SNAPSHOT. With this workaround, you can add a <force>true</force> xml node to the pom file which will add this dependency to the forced modules list which gradle uses to force specific versions on resolving. A good example for this would be our shop package. The pom file of a game would have an entry called core-shop in version 1.0. The core-shop has a dependency to the core-payment package 1.0. If I now want to work on the payment package, I normally need to change the dependency definitions of the shop package to tell it that it now has a dependency to a SNAPSHOT version. Being able to force a snapshot, I can just add the core-payment package in version 1.1-SNAPSHOT to the pom and the resolver will not resolve the 1.0 anymore but the 1.1-SNAPSHOT. The forced packages part (shown below) is the solution for this problem.

configurations {
    unityDependencies
    buildToolTemp
}
 
configurations.all {
 
    resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
 
    def forcedPackages = []
    def xmlPom = parsePomFile()
    xmlPom.dependencies.dependency.each
    {
        if(it.artifactId.text() != "CoreBuildTool")
        {
            if(it.force == "true")
            {
                forcedPackages.add("${it.groupId.text()}:${it.artifactId.text()}:${it.version.text()}")
            }
        }
    }
 
    resolutionStrategy.forcedModules = forcedPackages
}

At the dependencies resolutions phase, we are iterating through our dependencies listed in the pom file and store them in different configurations. As we need to handle the dependency resolver differently, we check if the version in the pom file differs from the one in the gradle properties file. If so, we store this dependency in its own configuration.

dependencies
{  
    def xmlPom = parsePomFile()
    project.dependencies
    {
        xmlPom.dependencies.dependency.each
        {
            if("${it.groupId.text()}" == "InnoCore" && "${it.artifactId.text()}" == "CoreBuildTool" && "${it.version.text()}" != props.version)
            {
                buildToolTemp "${it.groupId.text()}:${it.artifactId.text()}:${it.version.text()}"
            }
            else
            {
                unityDependencies "${it.groupId.text()}:${it.artifactId.text()}:${it.version.text()}"
            }
        }
    }
 
 
    compile gradleApi()
    compile localGroovy()
}

The clean folders task deletes all previous downloaded dependencies that are stored inside of Unity, to ensure we don’t have any old files there. The prepare update checks if there is a new version of the dependency resolver available, resolves and stores it in a temporary folder. The exchange of the old resolver and the newly downloaded is done by the Unity C# part afterwards.

task cleanAllFolder <<
{
    delete rootFolder + props.coreDependenciesFolder;
}
 
task prepareUpdate (type:Copy)
{
    def hasUpdate = false
    configurations.buildToolTemp.each
    {
        if(it.getName().reverse().take(3).reverse() == "zip")
        {
            from zipTree(it)
            into file(rootFolder + "/" + props.tempUpdateToolFolder)
        }
        else
        {
            from it
            into file(rootFolder + "/" + props.tempUpdateToolFolder)
        }
        hasUpdate = true
    }
}

The following tasks resolves the dependencies and copy the content of the artifacts placed in the gradle cache folder to Unity. We check for the file type here to directly place it in the correct subfolder if needed. The unzip All always depends on the clean folder task to ensure everything is cleaned up before we start to copy our dependencies over.

task unzipAll(type: GradleBuild, dependsOn: cleanAllFolder){
 
    configurations.unityDependencies.each
    {
        def depNameSplits = new ArrayList(Arrays.asList(it.getName().toString().split("\\.")))
        def ext = depNameSplits.pop();
         
        def depName = depNameSplits.join('.');
 
        if(ext == "zip")
        {
            copyZip( zipTree(it), file(rootFolder+"/"+props.coreDependenciesFolder))
        }
        else if(ext == "aar" || ext == "jar")
        {
            def dirPath = depName + "/Plugins/Android"
            new File(dirPath).mkdirs();
            copyFile(it, file(rootFolder + "/" + props.coreDependenciesFolder + "/" + dirPath))
        }
        else if(ext == "a" || ext == "framework")
        {
            def dirPath = depName + "/Plugins/iOS"
            new File(dirPath).mkdirs();
            copyFile(it, file(rootFolder + "/" + props.coreDependenciesFolder + "/" + dirPath))
        }
        else
        {
            println it
            copyFile(it, file(rootFolder + "/" + props.coreDependenciesFolder))
        }
    }
}

Every delivered package that needs some post steps after resolving contains a gradle script called installPackage. With the following part of the code, we search for this gradle script inside of the copied packages and execute a task there called installPackage. We mainly use this approach to copy classes that derive from ScriptableObject and MonoBehaviour over to a specified folder which is checked in into the games repository (including the meta file, so Unity can hold its reference to it).

task installPackage() {
    mustRunAfter "unzipAll"
}
 
installPackage << {
 
    File srcDir = file(rootFolder + "/" + props.coreDependenciesFolder)
    FileCollection collection = files { srcDir.listFiles() }
 
    collection.each { f ->
         
        if( f.isDirectory()){
             
            FileCollection packCont = files{ file(f).listFiles() }
            packCont.each { p ->
 
                if(p.name == "installPackage")
                {
                    def ppath = p.getPath()
                     
                    Task t = tasks.create(name: "installP-$ppath", type:GradleBuild) {
                        buildFile = "$ppath/installPackage.gradle"
                        tasks = ['installPackage']
                    }
                    t.execute()
                }
            }
        }
    }
}

Takeaway

This simple but stable solution provided everything we needed to have a convenient to use Unity dependency resolver which is in stable use by our games since 1 1/2 years now. With our newer solution which is in beta phase currently, we more tried to focus on bundling informations and collaboration directly in Unity. We tried to visualise as much as possible. A window now contains every information about our packages that a developer needs (list of all packages, documentation, changelog, versions to select, etc.). The Unity user is also able to select the source and the result of a resolve, which means he could build a local folder as a dependency to a dll or just symlink a repository into Unity to directly work on packages and much more. But more on that topic in one of the next articles.

If you have any questions, feel free to ask me at any time: https://www.linkedin.com/in/alexander-siemer-schmetzke-4654183b/