Running push-to-cloud to load a database dump file to Neo4j Aura

With the ability to download a database Dump from Aura instances, you have another tool along with Backup/Restore to manage your data and use it with flexibility in AuraDB and on-premise. 

In this case, we want to download a database Dump from an existing AuraDB instance and load it to another (or it could be the same) instance.

This way, you can decide to roll back your AuraDB Instance to a previous state (note that a dump is a snapshot and not a full backup) or decide you need to do some development and test on a duplicate of your current AuraDB Instance etc... 

The procedure can be applied to both AuraDB and AuraDS instances.

This article explains the procedure for using neo4j-admin push-to-cloudcommand available in local installations of Neo4j 4 and earlier. If you are using a 5.x version of Neo4j, please refer to the Using 'neo4j-admin database upload' in Neo4j 5.x to load a database dump to Neo4j Aura article.

Important Note on compatibility between different source and destination versions of Neo4j:
Neo4j 5.x instances(Destination) are backwards compatible and can accept backups from Neo4j 4.x and Neoj 5.x (Source).
Neo4j 4.x instances(Destination) are not forward compatible. 
i.e.

• You can import backups from Neo4j 5.x and 4.x instances into a Neo4j 5.x instance.
• You can import backups from Neo4j 4.x instances into a Neo4j 4.x instance.
• cannot import backups from Neo4j 5.x instances into 4.x. 

The process can be split into five steps:

1. Download a database Dump
2. Download the Export (a.k.a Dump file) locally to your machine.
3. Verify/Prepare your local Neo4j installation has the neo4j-admin tool.
4. Run the push-to-cloud command.
5. Monitor the status and availability of your Aura Instance during the upload.

1 - Download a database Dump (also called an Export).

Click the AuraDB Instance name in the Aura console and open up the SNAPSHOTS tab and download the dump file. 

2 - Download the Export (a.k.a Dump file) locally to your machine.

Download time is dependent on the dump file size (compressed datastore size) and of course your connection speed.
Note that the file will ultimately be saved as graph.db.dump locally (you can rename it but keep the file extension)

Note for AuraDS instances:
Snapshots from AuraDS instances will be a tar file that contains the instance's dump and GDS metadata.
Extract the .tar archive to get the 'neo4j.dump' file.

3 - Verify/Prepare your local Neo4j installation has the neo4j-admin tool.

You will need a Neo4j installation. the easiest option is to use the neo4j Desktop to achieve this as the push-to-cloud jar file is present by default.

If you are using Neo4j Desktop, access the settings from a 5.x database and select Terminal. This will open the terminal directly into the database's NEO4J_HOME directory.

To upload a dump file from other on-premise installations, you will need push-to-cloud (it should be present), please be sure to use the correct version of the binaries for the Neo4j version you are running or download them if required.

a - Dump file from a Neo4j v3.5.x  (or v3.6.x in Aura)

Please download the push-to-cloud binaries from here:
https://dist.neo4j.org/neo4j-push-to-cloud-3.5.23.jar

b - Dump file from a Neo4j v4.x (or Aura 4)

Please download the push-to-cloud binaries from here: 
For 4.0.x :https://dist.neo4j.org/neo4j-push-to-cloud-4.0.9.jar

For 4.1.x: https://dist.neo4j.org/neo4j-push-to-cloud-4.1.4.jar 

For 4.2.x: https://dist.neo4j.org/neo4j-push-to-cloud-4.2.0.jar 

The root/base directory of your Neo4j installation is referred as NEO4J_HOME  and you should place the binaries in <NEO4J_HOME>/lib
To verify if push to cloud is present and can run, execute:
<NEO4J_HOME>/bin/neo4j-admin help push-to-cloud

4 - Run the push-to-cloud command

Now that everything is ready, we can run the command to upload the dump.
We will need the AuraDB Instance URI (neo4j+s//<your_dbid>.databases.neo4j.io) as can be seen in the Aura console and your password (this is the password to the AuraDB Instance - not your Aura Console credentials). If you intend on creating a new AuraDB Instance, you need to do so now before you can run the command. 

The minimal set you will need to provide is (this means you will need to provide further info in interactive mode, like the username and password and confirmation you want to overwrite the content)

push-to-cloud --bolt-uri=neo4j+s://<dbid>.databases.neo4j.io --dump=/path/to/dumpfile/graph.db

Example:

Note: in this example, we have included the password on the command line but you can set it via environment variables (NEO4J_USERNAME, NEO4J_PASSWORD) or use the interactive mode for better security

user@hostname:installation-3.5.18$ bin/neo4j-admin push-to-cloud \

--bolt-uri=neo4j+s://ac43ded9.databases.neo4j.io \

--dump=/home/username/Downloads/graph.db.dump \

 --username=neo4j \

--password='U1JQFOX5_wgdwptm41PkHLekUBtDZ0y_7QX51RQuncU' \

--overwrite=true 

Upload
.................... 10%
.................... 20%
.................... 30%
.................... 40%
.................... 50%
.................... 60%
.................... 70%
.................... 80%
.................... 90%
.................... 100%
We have received your export and it is currently being loaded into your AuraDB instance.
You can wait here, or abort this command and head over to the console to be notified of when your AuraDB Instance is running.
Import progress (estimated)
.................... 10%
.................... 20%
.................... 30%
.................... 40%
.................... 50%
.................... 60%
.................... 70%
.................... 80%
.................... 90%
.................... 100%
Your data was successfully pushed to AuraDB and is now running.
It is safe to delete the dump file now: /home/username/Downloads/graph.db.dump

5 - Monitor the status and availability of your AuraDB Instance in the Aura console

While the dump file is uploading the AuraDB Instance is still marked as Running.

Then the AuraDB Instance will display Loading

And when it is complete you will see Running status again

Additional notes

If you use the --database switch the command assumes this is for the local database instance and will search in the matching dump location

For full options of the push-to-cloud command see the help:

 user@hostname:$ bin/neo4j-admin help push-to-cloud
usage: neo4j-admin push-to-cloud [--database=<name>]
[--dump=</path/to/my-neo4j-database-dump-file>]
[--dump-to=</path/to/dump-file-to-be-created>]
--bolt-uri=<bolt+routing://mydatabaseid.databases.neo4j.io>
[--verbose] [--username=<neo4j>]
[--password=<mYs3cr3tPa$$w0rd>]
[--overwrite=<true/false>]

environment variables:
NEO4J_CONF Path to the directory which contains neo4j.conf.
NEO4J_DEBUG Set to anything to enable debug output.
NEO4J_HOME Neo4j home directory.
HEAP_SIZE Set JVM maximum heap size during command execution.
Takes a number and a unit, for example, 512m.

Push your local database to a Neo4j AuraDB instance. The database must be shut down
in order to take a dump to upload. The target location is your Neo4j AuraDB Bolt
URI. You will be asked your Neo4j Cloud username and password during the
push-to-cloud operation.

options:
--database=<name>
Name of database. [default:graph.db]
--dump=</path/to/my-neo4j-database-dump-file>
Path to an existing database dump for upload. This arugment cannot be used
together with --database. [default:null]
--dump-to=</path/to/dump-file-to-be-created>
Target path for dump file. Used in combination with the --database
argument. [default:null]
--bolt-uri=<bolt+routing://mydatabaseid.databases.neo4j.io>
Bolt URI of target AuraDB Instance
--verbose
Enable verbose output.
--username=<neo4j>
Optional: Username of the target AuraDB Instance. Prompt
will ask for the username if not provided. Alternatively, NEO4J_USERNAME
environment variable can be used. [default:null]
--password=<mYs3cr3tPa$$w0rd>
Optional: Password of the target AuraDB Instance. Prompt
will ask for a password if not provided. Alternatively, NEO4J_PASSWORD
environment variable can be used. [default:null]
--overwrite=<true/false>
Optional: Overwrite the data in the target AuraDB Instance. [default:false]

user@hostname: