Code sync for hybrid development

In this page, we describe how to use Crafting's code sync feature for hybrid development.

Similar to port-forwarding feature, the code sync feature also allows you to combine the power of cloud with the familiarity of the development environments on local machine. But different from port-forwarding, you can further leverage the power of cloud machines to run the target service you are working on. This is especially useful when your are working on a large repo where build and run the service take a large amount of computational resources.

Specifically, in code sync mode for hybrid development:

• All services are running on cloud and not taking any local resources
• The target service (e.g. Service A), has source code on both local machine and a cloud workspace.
• The developer uses a desktop IDE to edit the source code of Service A locally. The IDE doesn't need to have remote development capability.
• As the developer edits the source code, it's synced on the background to the cloud workspace by tools integrated with Crafting CLI, so that the source code on the cloud workspace is kept same as the local source code.
• The developer can trigger remote build and restart the service after coding, and then test the end-to-end flow via Internet facing endpoints provided in Crafting sandbox.
• The run log can be viewed on Crafting log viewer or via Crafting CLI from local machine. Breakpoints can be set via port forwarding of a remote debug port.

Code sync tools provided with Crafting

Crafting provides a few alternatives for sync code between local and cloud workspace.

Rsync integration

Crafting integrates with rsync command to sync the file between local machine and cloud workspace.

$ cs rsync <LOCAL-PATH> <SANDBOX/WORKSPACE>:<REMOTE-PATH>

The command above runs rysnc once between the LOCAL-PATH on the local machine and REMOTE-PATH on the cloud workspace under the specified sandbox. The files and directory structure in the LOCAL-PATH will be synced to the remote, following the rsync semantics, which can be found by man rsync

Mutagen integration

An integration with mutagen is also provided by Crafting for continuously syncing the files.

$ cs mutagen <LOCAL-PATH> <SANDBOX/WORKSPACE>:<REMOTE-PATH>

The command above establishes a sync (by default a two-way sync) between the LOCAL-PATH on the local machine and REMOTE-PATH on the cloud workspace under the specified sandbox. The sync session is continuous and managed by mutagen, so that whenever a file change is detected from one side, it would copy the delta to the other side.

To further optimize the performance, you may want to ignore certain files like temp files generated while coding, please see mutagen's documentation for more details. To pass flags to mutagen from cs command, do it after --, e.g. cs mutagen /home/jack/backend jack/backend:/home/owner/backend -- --ignore-vcs

Via Git and auto-follow

Last but not least, a developer can just rely on the Git to sync the code between local and sandbox.

Crafting provides a convenient feature for the source code in the sandbox to auto-follow a specific branch in Git repo. With this feature, a developer can simply launch a sandbox and select the workspace to auto-follow the repo for a personal branch, and then whenever the code needs to be synced from local, a simple git push will do the job.

Please see Auto-follow code branch in sandbox for details.

Execute build and run command on sandbox from local machine

In the code sync hybrid development mode, we often need to run the build with the updated code on the cloud workspace and restart the running service to test. For that, we can simply use cs ssh to execute command on sandbox from a local terminal

$ cs ssh [-W SANDBOX/WORKSPACE] <command>

The above command executes the command on the specific workspace inside the sandbox. You can do something like cs ssh -W jack/backend "cd /home/owner/backend && ./build.sh" to build the code remotely.

Keep in mind that typically build and run commands are supposed to be preset in templates and ready to be run as cs build and cs restart. So something like cs ssh -W jack/backend "cd /home/owner/backend && cs build && cs restart" would do everything needed for build and restart.

Forward remote debug port for remote debugging with local IDE

When launching a process in the workspace using a remote debug agent, the listening port can be forwarded to local, using the cs portforward command, assume the port is 1234:

$ cs portforward -PF -L 1234:1234

The flag -PF disables auto-discovered port-forwarding rules. -L specifies local to remote port-forwarding. After that, launching a debugger in local IDE and connecting to localhost:1234 will allow us to debug the report process running in the sandbox.

Comparison for different models for developments

The following table compares the three models of development with Crafting sandbox: