Updated 2020-02-05 

Getting Started

This page carries on from a successful 👈 Quickstart into making it relevant to your usecase.

What is Allzpark?

It's an application launcher, similar to the one on the home screen of your phone or in the Start menu on Windows. The difference is that before actually launching the application, you are able to tailor the associated environment within which the applicatuon runs.

Target Audience

Visual Effects
Games
Software Development

For example...

Let's say you're working on a project using (1) Autodesk Maya, (2) Adobe Photoshop and (3) Pixologic Zbrush.

For Maya, you'd like to use various software libraries, Python scripts and binary plug-ins like Solid Angle's Arnold and Ziva Dynamics. However, each plug-in need to be of a particular version in order to play ball this version of Maya.

Over time, the number of projects grow, as do the individual requirements of each application, and many of the plug-ins and libraries end up sharing dependencies with each other.

So now you've got a network of interdependencies that all need to work with each other and wonder..

"How can I write my own software that run in this environment?"
"How can I make sure my software runs once deployed?"

That's where reproducible enironments come in handy and Allzpark can do for you.

Rez - an industry standard used by studios large and small - is the framwork with which you design this network and Allzpark is the shiny front-end your artists uses interface with it.

Studios using Rez

What are some similar projects?

Have a look at these, they solve a similar problem to Allzpark.

So then, why should you choose Allzpark? Odds are you already have most of what these solutions offer and aren't ready for a complete swap. Allzpark integrates with existing pipelines and works standalone. It'll also work the day you decide to transition from e.g. Shotgun to ftrack, safeguarding your investment in each individual component of your pipeline.

Here's how you would typically use Allzpark.

Boot machine
Boot Allzpark
Select profile
Boot application

For the purposes of this quick walkthrough, I'll assume you are part of a visual effects studio whereby "profile" means e.g. Alita and "application" means e.g. Blender.

What is a "profile"?

alita is the name given to a profile within which work is performed by multiple stakeholders towards a common goal. It typically consists of these components.

Name
Version
Requirements
Environment
Applications

Whereby an "application" is the software used within this profile, such as blender, photoshop and sublimetext.

The "requirements" indicate what software or libraries your profile depend on, such as python-3 or git-2.21 or arnold-6.

The "environment" are hand-crafted variables stored in the launched application. Variables you can later refer back to in the software you write to run within the context of that application. For example, PROJECT_NAME=alita is a relevant variable to add, to allow for the application and your software to identify which profile an application was launched in.

As you might have guessed, projects are versioned and we'll get into more about this and "packages" in general a little later.

What is an "application"?

blender is an application within which work is performed.

In Allzpark, the profile dictates what applications are available to the user, in order to faciliate a "data pipeline" being built around a pre-determined set of software and libraries.

Data pipeline?

A kind of "codified" workflow. For example, you use the same settings for whenever you export an image from Photoshop to your game engine. Rather than explicitly setting those each time, you make a script to turn the process into a single click. The same then applies to any kind of export and import of data in various applications, to and from various stakeholders in your company.

These are some examples of pre-made "scripts" - in the form of frameworks - suitable for use with Allzpark.

An application typically consists of these components.

Name
Version
Requirements
Environment

Notice that it isn't unlike a profile, and in fact not unlike any other software you'll encounter later on. These are both "packages" and we'll talk a lot more about what that is as we progress through this guide.

What is a "package"?

I've talked a lot about "packages" and how great they are, but what exactly are they and why should you care?

Without Rez and without Allzpark, software is typically installed "system-wide". That is, your Python install resides somewhere on your system and is automatically added to your system PATH, such that when you type..

python

From either Bash or PowerShell, this is the version of Python picked up.

A "package" is this, except rather than installing, in this case, Python onto your system, and adding it to your system PATH, it is installed into a portable folder which is added to PATH only when referenced.

rez env python
> python

A package can contain both files, environment variables but also arbitrary commands that execute whenever the package is referenced, which is how you distribute both applications like Python and libraries and, in the case of Allzpark, profiles.

Your first profile

You and I are going to embark on a new profile. Let's call it kingkong

Command Line

The way we'll establish this profile, and packages like it, is going to involve the command-line, so let's get comfortable with how it works.

I'll provide command-line instructions for both powershell and bash, to cover both Windows, Linux and MacOS users. You can follow along using either of the two flavours, but odds are the one pre-selected is the one you'll want to use.

$env:POWERSHELL="These lines are for powershell"

$var = "If you are on Windows, this is where we'll
        spend most of our time. It also applies to
        pwsh, the cross-platform PowerShell 6+."

echo These lines are for bash
echo typically used in MacOS and Linux

Now select a shell of your choice and let's get going.

JavaScript

If you aren't seeing any code, make sure you have JavaScript enabled. If this is a problem for you, let me know.

King Kong

Each profile requires a folder and a file called package.py.

mkdir ~/kingkong
cd ~/kingkong
@"
name = "kingkong"
version = "1.0.0"
build_command = False
"@ | Add-Content package.py
rez build --install

mkdir ~/kingkong
cd ~/kingkong
echo name = "kingkong" >> package.py
echo version = "1.0.0" >> package.py
echo build_command = False >> package.py
rez build --install

That's it, we've now got a brand new profile. Let's add it to Allzpark and see what it looks like.

$env:MY_PROFILES="kingkong"
allzpark --demo

export MY_PROFILES="kingkong"
allzpark --demo

We'll do a lot more of this as we go along, so don't worry if it doesn't quite make sense just yet.

Regarding MY_PROFILES

I've programmed this demo to take the environment variable MY_PROFILES into account, but we'll have a look later at how you can customise how projects are actually discovered either from disk, a production tracking system like Shotgun or arbitrary function you provide.

What we've learned

Let's take a moment to reflect on what we've accomplished so far.

We've gotten familiar with the rez command
We've authored a new Rez package from scratch
We've used rez build, one of many Rez sub-commands, to build and install a package
We've made Allzpark aware of this new profile package, via the MY_PROFILES environment variable.

Next we'll have a look at how to add an application to your profile, and how to actually make a new application from scratch.

Your first application

There isn't much we can do with a profile unless we've got an application, so let's add one.

Open kingkong/package.py in your favourite text editor
Edit it as follows.

name = "kingkong"
version = "1.0.1"
build_command = False

requires = [
    "~maya==2018.0.6",
    "~blender==2.80.0",
]

As you may have guessed, these are the requirements of this profile. That little squiggly ~ character ahead of maya indicates that this is a "weak" reference, which Allzpark interprets as application in this profile.

Pro tip

If you're already familiar with Rez and think to yourself "This isn't very flexible", you're right. Looking for applications in the requirements section of a package is a default you can customise later via the allzparkconfig.py:applications_from_package() function.

We'll talk more about requirements next, let's install this package and launch Allzpark.

cd kingkong
rez build --install
allzpark --demo

allzpark_kingkong

Pro tip

You'll notice the order in which you specified the applications are respected in the GUI.

An application package

Ok, but we didn't really create an application so much as just add an existing one to the profile. Let's create a new application from your OS and add that to the profile too.

You can pick any application you'd like, for the purposes of this guide I'll make a package for a text editor.

mkdir ~/texteditor
cd ~/texteditor
@"
name = "texteditor"
version = "1.5.0"
build_command = False
"@ | Add-Content package.py
rez build --install

mkdir ~/texteditor
cd ~/texteditor
echo name = "texteditor" >> package.py
echo version = "1.5.0" >> package.py
echo build_command = False >> package.py
rez build --install

You'll notice the similarity to creating a profile and that's no coincidence. These are both Rez "packages". But there's something missing.

Unlike a profile, an application must either reference an executable on disk, or encapsulate this executable into the package. We'll get into encapsulating files with a package a little later, for now let's have a look at how to reference a file on disk.

texteditor/package.py

name = "texteditor"
version = "1.5.1"
build_command = False

def commands():
    import os
    global alias

    if os.name == "nt":
        alias("texteditor", r"notepad")
    elif os.name == "darwin":
        alias("texteditor", r"textedit")
    else:
        alias("texteditor", r"gedit")

The commands() function is called whenever your package is referenced, and alias() creates a command you can use from within a context. We're able to leverage regular Python imports and commands here as well, which is how we detect the running operating system.

About assumptions

One of the issues with referencing system software like this is, well, how can we be sure these actually exist on the target operating system? For Windows it's a given, but what about Linux? We'll see this problem crop up again later, and is in fact already an issue with the pre-existing demo packages for Maya and Blender.

For the purposes of this guide, these assumptions are fine and I'll show you later how you can avoid making them, and at what cost.

Let's try this out.

cd texteditor
rez build --install
rez env texteditor --paths $(allzparkdemo --packages)
> texteditor

This is the equivalent command-line procedure to what Allzpark is doing when you launch an application. Don't worry too much about what rez env actually does right now, we'll talk a lot more about it later. Let's try this out in Allzpark too.

kingkong/package.py

name = "kingkong"
version = "1.0.2"
build_command = False

requires = [
    "~maya==2018.0.6",
    "~blender==2.80.0",
    "~texteditor==1.5.1",
]

cd kingkong
rez build --install
allzpark --demo

And presto!

Pro tip

Notice that our terminal doesn't yet have an icon, and its name is all lowercase and plain. We'll address this next, in the Payload chapter.

What we've learned

You made it! Let's reflect on what we've learned so far.

Creating a new application is not unlike creating a new profile
Packages have a commands() function you can use to "bootstrap" an environment with custom commands
There's pros and cons to referencing system software, like a text editor.

Your first payload

We've managed to make a new profile, and an application and we're just about ready to start developing the next King Kong movie.

But there is something else missing. For the purposes of this chapter, I will assume you are developing King Kong using Autodesk Maya, but the same applies to just about any application.

Let's start by adding some files to your package.

Adding some files

kingkong/
    resources/
        icon.png

I'm going to use this image, but you can use anything you've got lying around, so long as it's a .png file of reasonable size and aspect ratio.

Then I'll update our package.py as well.

package.py

name = "kingkong"
version = "1.0.3"
build_command = "python {root}/install.py"  # 1

_data = {                                   # 2
    "label": "King Kong",
    "icon": "{root}/resources/icon.png",
}

requires = [
    "python-2.7,<4",                        # 3
    "~maya==2018.0.6",
    "~blender==2.80.0",
    "~texteditor==1.5.1",
]

Let's walk through these changes.

We first update out build_command to call on a script, the script we'll use to copy files from your package folder to the build/ folder which is later installed.
Next we add _data which isn't related to Rez, but rather arbitrary data provided to a package. You can add any number of variables, but it is good practice to avoid clashing with built-in variables by prefixing it with _. In this case, Allzpark is programmed to look for this variable to find label and icon.
We add python-2.7,<4 to our list of requirements, without ~ this time.

Here's what the files look like.

install.py

# This script is called on `rez build`
import os
import shutil

print("Running install.py...")
root = os.path.dirname(__file__)
build_dir = os.environ["REZ_BUILD_PATH"]
install_dir = os.environ["REZ_BUILD_INSTALL_PATH"]

print("Copying payload to %s.." % build_dir)
shutil.copytree(
    os.path.join(root, "resources"),
    os.path.join(build_dir, "resources"),
    ignore=shutil.ignore_patterns("*.pyc", "__pycache__")
)

if int(os.getenv("REZ_BUILD_INSTALL")):
    # This part is called with `rez build --install`
    print("Installing payload to %s..." % install_dir)
    shutil.copytree(
        os.path.join(build_dir, "resources"),
        os.path.join(install_dir, "resources"),
    )

Now let's try and build our package once more.

cd kingkong
rez build --install --clean
allzpark --demo

--clean will erase any existing build/ directory, such that you can build over and over and over.

New version

Make sure you increment your version each time you install, or implement the appropriate clean-up functionality in your install.py script, as otherwise it may throw an exception about not being able to write into a directory that already has files in it.

Generally, it's fine to overwrite packages you install locally; we'll talk about how local differs from a "release" a little later, and how you can employ continuous integration and write-protect your public release folder for an additional level of security.

allzpark_kingkonglabel

What we've learned

High five! ✋ You've now learnt how to include files with your package, this is what we'll use to distribute software like Python libraries along with profile-specific files like the icon we've seen but also files related to potential applications you use, such as Maya script files or shelves.

But what if you wanted to include a Python library and call that from an application? How can the application be made aware of what you've included in your package? This is where the environment comes in, and we'll have a look at that next.

Your first environment

So your profile has got some custom data, that's perfect. Now let's add some metadata as well.

name = "kingkong"
version = "1.0.5"
build_command = "python {root}/install.py"

_data = {
    "label": "King Kong",
    "icon": "{root}/resources/icon.png",
}

requires = [
    "python-2.7,<4",
    "~maya==2018.0.6",
    "~blender==2.80.0",
    "~texteditor==1.5.1",
]

def commands():
    global env

    env["PROJECT_ID"] = "12"
    env["PROJECT_NAME"] = "kingkong"
    env["PROJECT_FRAMERATE"] = "25"
    env["PROJECT_TAGS"] = "awesome,great,best"

    env["PYTHONPATH"].prepend("{root}/python")

The global variable env is shared amongst each of your packages, passed from one to the next, in the order they are resolved.

Pro tip

Packages are resolved in the order they are required. That is, if Alita requires Maya, then Maya's commands() function is called first.

As you've noticed, we can use this as both a dictionary and class with a .prepend() member. Prepend automatically treats the variable as a PATH-like variable and .prepend() in this case prepends a given value to that variable. There's also .append() for the opposite effect. Whether to append or prepend depends on your use case.

# Prepend
PYTHONPATH="c:\packages\alita\1.0\python;c:\packages\maya\2018.0\python"

# Append
PYTHONPATH="c:\packages\maya\2018.0\python;c:\packages\alita\1.0\python"

# Prepend
PYTHONPATH=/packages/alita/1.0/python:/packages/maya/2018.0/python

# Append
PYTHONPATH=/packages/maya/2018.0/python:/packages/alita/1.0/python

Ok, that's great, the package now prepends "{root}/python", let' actually create that folder now.

Adding a new file

kingkong/
    resources/
        icon.png
    python/
        kingkong.py

In my file, I'll put something like this.

kingkong/python/kingkong.py

def king():
    print("kong!")

We'll also need to update our install.py to include this file.

Updating our build command

kingkong/install.py

# This script is called on `rez build`
import os
import shutil

print("Running install.py...")
root = os.path.dirname(__file__)
build_dir = os.environ["REZ_BUILD_PATH"]
install_dir = os.environ["REZ_BUILD_INSTALL_PATH"]
dirs = ["resources", "python"]

print("Copying payload to %s.." % build_dir)
for dirname in dirs:
    shutil.copytree(
        os.path.join(root, dirname),
        os.path.join(build_dir, dirname),
        ignore=shutil.ignore_patterns("*.pyc", "__pycache__")
    )

if int(os.getenv("REZ_BUILD_INSTALL")):
    # This part is called with `rez build --install`
    print("Installing payload to %s..." % install_dir)
    for dirname in dirs:
        shutil.copytree(
            os.path.join(build_dir, dirname),
            os.path.join(install_dir, dirname),
        )

Now let's build and run.

cd kingkong
rez build --install --clean
allzpark --demo

allzpark_king

Bonus Chapter

As you're learned, writing that install.py script is rather tedious, and you'll need one of these for every package you create. Wouldn't it be great if you could make a generic install script that you could reuse across each of your projects?

Well, yes, yes you can!

name = "kingkong"
version = "1.0.5"
build_command = "python -m rezutil build {root}"

_data = {
    "label": "King Kong",
    "icon": "{root}/resources/icon.png",
}

requires = [
    "python-2.7,<4",
    "rezutil-1",
    "~maya==2018.0.6",
    "~blender==2.80.0",
    "~texteditor==1.5.1",
]

def commands():
    global env

    env["PROJECT_ID"] = "12"
    env["PROJECT_NAME"] = "kingkong"
    env["PROJECT_FRAMERATE"] = "25"
    env["PROJECT_TAGS"] = "awesome,great,best"

    env["PYTHONPATH"].prepend("{root}/python")

Notice the addition of rezutil, and that I've removed any reference to install.py, that you can go ahead and delete. This is an example of using a Rez package to build another Rez package and is a really common technique for situations just like this.

Let's build the project to confirm we're getting the exact same result.

cd kingkong
rez build --install --clean

Et voila!

Pro tip

Also notice that we didn't update the version this time, because built-in to rezutil is handling overwrites along with retries for when writing to an unreliable network resource.

You can find this mystical rezutil package here, as well as on your local harddrive found here:

start $(allzparkdemo --packages)

allzparkdemo --packages

Pro tip

The source package can be found one level up, in the dev/ directory.

Bonus Bonus Chapter

One final note about the rezutil package; you'll notice that because we've included it as a requirement to the kingkong profile, the package is also made visible to any application started from within this profile. In this case however, this package is only really relevant during package build.

We can work around this by making it a private requirement.

name = "kingkong"
version = "1.0.5"
build_command = "python -m rezutil {root}"
private_build_requires = ["rezutil-1"]

_data = {
    "label": "King Kong",
    "icon": "{root}/resources/icon.png",
}

requires = [
    # "python-2.7,<4",
    # "rezutil-1",
    "~maya==2018.0.6",
    "~blender==2.80.0",
    "~texteditor==1.5.1",
]

def commands():
    global env

    env["PROJECT_ID"] = "12"
    env["PROJECT_NAME"] = "kingkong"
    env["PROJECT_FRAMERATE"] = "25"
    env["PROJECT_TAGS"] = "awesome,great,best"

    env["PYTHONPATH"].prepend("{root}/python")

I also removed the requirement for Python, as this is also only really relevant during build.

What we've learned

Woho! You're a fully certified Rez environment artist!

A package can modify its environment
PATH-like variables have a convenient prepend and append, more on that here
Variables have access to special variables, like {root}, more about that here
You can use Rez packages to build Rez packages (neat!)

Debugging

As you author packages and configure your profile, you'll run into issues; either from Rez or Allzpark itself.

When an error occurs, either:

Inspect the Console tab for clues
Replicate the problem in the command-line

The Console does its best for forward issues coming from Rez, such as syntax errors in your package, unreferenced environment variables, mismatching requirements and so on. But the command-line gives you one superior advantage.

Isolating a context

By isolating, I mean that you can establish a context using just a handful - or even a single - request.

rez env my_broken_package
# Error here

This is how you can narrow down whether an error is due to a particular package, or a package in combination with others.

rez env packageA -- exit  # No error
rez env packageB -- exit  # No error
rez env packageA packageB -- exit
# Error

With Advanced Controls visible, you can find the complete command Allzpark uses to establish a context visually at the bottom of the applications view.

Configuring Allzpark

The --demo flag is responsible for determining how and what profiles to visualise in Allzpark, let's have a look at how we can go independent from --demo with our won custom allzparkconfig.py

Work in progress

Let me know if you would like me to flesh out this section here, or help out by contributing to the project!

What's Next?

From here, you are able to craft packages of any kind, both profiles, applications and libraries along with linking them together via requires (and let's not forget private_build_requires!). At this point, you're ready to head out into the wild.

Package Examples
Advanced Profile Example
Allzpark Tutorials
Rez Wiki
Rez Tutorials
Rez Chat (Ask for invite on mailing list)
Rez Mailing List

Something else on your mind? Let me know!

bleeding vs. nerdvegas Rez

In this project we use bleeding-rez which is a flavour of Rez better suited for profiles like the one we've made here, but the two projects are nearly identical. When asking for help, it's safe to assume that any issue you're having is relevant to both projects; however it can be worth mentioning that you are using bleeding-rez, rather than nerdvegas/rez.