What will happen when we delete a node or a relationship in Neo4j database? After you get a response message from Neo4j database that tell you the node or relationship has been successfully delete, in reality, at the backend, it just means that the storage is marked as "deleted" and will be used later when new nodes or relationship is created. It also means after the deletion, Neo4j database is still managing the storage space, the total storage space of the database will not shrink at this point.
If you want to compress the db to remove those records and optimise the space used, they need to use neo4j-admin copy (or, if on 3.5.x or below, the store-utils).
|07:55:40|Neo4j:[import]> neo4j-admin copy --to-database=testdumpcopy1 --from-database=testdump --force --verbose
neo4j 4.2.4
VM Name: Java HotSpot(TM) 64-Bit Server VM
VM Vendor: Oracle Corporation
VM Version: 11.0.9+7-LTS
JIT compiler: HotSpot 64-Bit Tiered Compilers
VM Arguments: [-XX:+UseParallelGC, -Xms4096m, -Xmx8192m, -Dfile.encoding=UTF-8]
Starting to copy store, output will be saved to: /Users/fyao/softwares/neo4j-enterprise-4.2.4/logs/neo4j-admin-copy-2021-06-08.09.56.48.log
2021-06-08 16:56:49.573+0000 INFO [StoreCopy] ### Copy Data ###
2021-06-08 16:56:49.573+0000 INFO [StoreCopy] Source: /Users/fyao/softwares/neo4j-enterprise-4.2.4/data/databases/testdump (page cache 8m)
2021-06-08 16:56:49.573+0000 INFO [StoreCopy] Target: /Users/fyao/softwares/neo4j-enterprise-4.2.4/data/databases/testdumpcopy1 (page cache 8m)
2021-06-08 16:56:49.574+0000 INFO [StoreCopy] Empty database created, will start importing readable data from the source.
2021-06-08 16:56:53.700+0000 INFO [o.n.i.b.ImportLogic] Import starting
Nodes, started 2021-06-08 16:56:56.488+0000
[*Nodes:0B/s 30.86MiB-------------------------------------------------------------------------]6.15M ∆ 450K
Done in 11s 137ms
Prepare node index, started 2021-06-08 16:57:07.781+0000
[*DETECT:39.12MiB-----------------------------------------------------------------------------]7.07M ∆60.0K
Done in 766ms
Relationships, started 2021-06-08 16:57:08.554+0000
[*Relationships:?? 39.12MiB-------------------------------------------------------------------] 0 ∆ 0
Done in 6ms
Node Degrees, started 2021-06-08 16:57:10.616+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
Relationship --> Relationship 1/1, started 2021-06-08 16:57:10.675+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 1ms
RelationshipGroup 1/1, started 2021-06-08 16:57:10.698+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 12ms
Node --> Relationship, started 2021-06-08 16:57:10.725+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 12ms
Relationship <-- Relationship 1/1, started 2021-06-08 16:57:10.774+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
Count groups, started 2021-06-08 16:57:12.030+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
Gather, started 2021-06-08 16:57:12.102+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
Write, started 2021-06-08 16:57:12.144+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
Node --> Group, started 2021-06-08 16:57:12.175+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
Node counts and label index build, started 2021-06-08 16:57:13.624+0000
[*>-----------------------------------------|LABEL INDEX-----------------|COUNT:24.50MiB------]2.17M ∆2.17M
Done in 358ms
Relationship counts and relationship type index build, started 2021-06-08 16:57:14.019+0000
[*>:??----------------------------------------------------------------------------------------] 0 ∆ 0
Done in 0ms
IMPORT DONE in 21s 975ms.
Imported:
2162447 nodes
0 relationships
18224200 properties
Peak memory usage: 39.12MiB
2021-06-08 16:57:15.678+0000 INFO [o.n.i.b.ImportLogic] Import completed successfully, took 21s 975ms. Imported:
2162447 nodes
0 relationships
18224200 properties
2021-06-08 16:57:18.384+0000 INFO [StoreCopy] Import summary: Copying of 6153114 records took 28 seconds (219754 rec/s). Unused Records 3990667 (64%) Removed Records 0 (0%)
2021-06-08 16:57:18.384+0000 INFO [StoreCopy] ### Extracting schema ###
2021-06-08 16:57:18.384+0000 INFO [StoreCopy] Trying to extract schema...
2021-06-08 16:57:18.551+0000 INFO [StoreCopy] ... found 4 schema definitions. The following can be used to recreate the schema:
2021-06-08 16:57:18.552+0000 INFO [StoreCopy]
CALL db.createUniquePropertyConstraint( 'constraint_user_instagramShadowToken', ['User'], ['instagramShadowToken'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]} );
CALL db.createUniquePropertyConstraint( 'constraint_user_shadowtoken', ['User'], ['shadowToken'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]} );
CALL db.createUniquePropertyConstraint( 'constraint_user_id', ['User'], ['id'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]} );
CALL db.createIndex('index_country_code', ['User'], ['countryCode'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]})
2021-06-08 16:57:18.553+0000 INFO [StoreCopy] You have to manually apply the above commands to the database when it is started to recreate the indexes and constraints. The commands are saved to /Users/fyao/softwares/neo4j-enterprise-4.2.4/logs/neo4j-admin-copy-2021-06-08.09.56.48.log as well for reference.
Here is a comparison about the space usage before and after the neo4j-admin copy execution.
In your Neo4j browser window, you can type :sysinfo , and it will show you the store size of your database. You can see that the store size shrink from 8.67GB to 503.78MB.
more details can be found here:
https://neo4j.com/docs/operations-manual/current/backup-restore/copy-database/
Comments
0 comments
Please sign in to leave a comment.