By Theofilos Ioannidis (tioannid [at] di [dot] uoa [dot] gr), created on , last updated on
Containers are not ideal for benchmarking purposes similar to the one GeoRDFBench Framework performs, because they do not allow clearing system caches. The reason for this is that:
Therefore, in the following example, although the user can verify that the experiments run properly and results are correctly calculated and reported, the COLD cache response times will not be accurate. However, for experiments that do not require COLD cache response time measurements, e.g., macro benchmark scenarios, response times should be accurate enough for drawing basic conclusions.
This example, features:
For this example, we will use an Ubuntu 22.04 (jammy) host machine, with 32 GB of DDR4-2400MHz memory, an Intel i7-7700 CPU (8 executing threads=4 cores * 2 threads/core), a Samsung SSD NVMe 970 EVO Plus 500GB system disk and a 2TB HDD 2.5" SATA III 7200rpm for data disk.
The docker image is stored in the Github registry under tioannid/geordfbench/strabonvirtuoso/synthetic. We assume that the current directory is /data/TEST. Then we issue the following commands, which pull the image from the registry, and verify its presence in the docker image list:
tioannid@tioannid-OMEN-by-HP-Laptop-17-an0xx:/data/TEST$ docker pull ghcr.io/tioannid/geordfbench/strabonvirtuoso/synthetic:latest
latest: Pulling from tioannid/geordfbench/strabonvirtuoso/synthetic
cbe3537751ce: Pull complete
c8007fac92e2: Pull complete
9b87307ed344: Pull complete
a84381c40ef4: Pull complete
a917b90f6482: Pull complete
9db462bae9ce: Pull complete
79ea1e1fa0e2: Pull complete
3da55c090a3a: Pull complete
7825797420ee: Pull complete
a9157fc8a24f: Pull complete
f2bfe66e6462: Pull complete
Digest: sha256:7a5fd9d71ab250a5eacd048abfb89ef41f10333a45eb7efbaf9bba5cc642208f
Status: Downloaded newer image for ghcr.io/tioannid/geordfbench/strabonvirtuoso/synthetic:latest
ghcr.io/tioannid/geordfbench/strabonvirtuoso/synthetic:latest
tioannid@tioannid-OMEN-by-HP-Laptop-17-an0xx:/data/TEST$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ghcr.io/tioannid/geordfbench/strabonvirtuoso/synthetic latest 00c8da201321 7 hours ago 3.54GB
We start a container named mybench_2 from this image with the following command:
tioannid@tioannid-OMEN-by-HP-Laptop-17-an0xx:/data/TEST$ docker run -e POSTGRES_PASSWORD=postgres -p 5432:5432 --hostname NUC8i7BEH --cpus="4" --memory="24g" --memory-swap="24g" --mount "type=bind,src=`pwd`,target=/src" \
--name mybench_2 -d ghcr.io/tioannid/geordfbench/strabonvirtuoso/synthetic
da8c108d02c66d17bf77387403a64d9e28ea1d0bc668c2a8052d52ee5f38cd0f
tioannid@tioannid-OMEN-by-HP-Laptop-17-an0xx:/data/TEST$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
da8c108d02c6 ghcr.io/tioannid/geordfbench/strabonvirtuoso/synthetic "tail -f /dev/null" 6 seconds ago Up 5 seconds 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp mybench_2
This command, launches the mybench_2 container, while defining:
In order to better understand this experiment, we need to review the differences between the original Geographica 2 micro execution spec and the modified copy fast micro execution spec. We start by connecting to the mybench_2 running container with a terminal and issue the command:
tioannid@tioannid-OMEN-by-HP-Laptop-17-an0xx:/data/TEST$ docker exec -it mybench_2 /bin/bash
root@NUC8i7BEH:/data# cd geordfbench/json_defs/executionspecs/
root@NUC8i7BEH:/data/geordfbench/json_defs/executionspecs# diff --color microESoriginal.json microESoriginal_fast.json
4,5c4,5
< "COLD" : 3,
< "WARM" : 3
---
> "COLD" : 1,
> "WARM" : 1
7,8c7,8
< "maxDurationSecsPerQueryRep" : 1800,
< "maxDurationSecs" : 3600,
---
> "maxDurationSecsPerQueryRep" : 180,
> "maxDurationSecs" : 360,
12,13c12,13
< "clearCacheDelaymSecs" : 5000
< }
\ No newline at end of file
---
> "clearCacheDelaymSecs" : 2000
> }
...The above coloured diff output shows that we chose to:
We start the experiments by issuing the command:
root@NUC8i7BEH:/data/geordfbench/json_defs/executionspecs# cd /data; ./startUpScript.sh
* Starting PostgreSQL 14 database server [ OK ]
...
CREATE DATABASE
ALTER DATABASE
You are now connected to database "geographica3" as user "postgres".
...
CREATE TABLE
ALTER TABLE
CREATE SEQUENCE
...
GRANT
GRANT
...
[INFO] Installing /data/util_misc/virt_rdf4j.jar to /root/.m2/repository/com/virtuoso/openlink/virt_rdf4j/3.7.6/virt_rdf4j-3.7.6.jar
[INFO] Installing /tmp/mvninstall14865250812882637003.pom to /root/.m2/repository/com/virtuoso/openlink/virt_rdf4j/3.7.6/virt_rdf4j-3.7.6.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.450 s
[INFO] Finished at: 2023-12-17T10:08:36Z
[INFO] ------------------------------------------------------------------------
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.4:install-file (default-cli) @ standalone-pom ---
[INFO] Installing /data/util_misc/virtjdbc4_2.jar to /root/.m2/repository/com/virtuoso/openlink/virtjdbc4_2/4.2/virtjdbc4_2-4.2.jar
[INFO] Installing /tmp/mvninstall8325055862404070817.pom to /root/.m2/repository/com/virtuoso/openlink/virtjdbc4_2/4.2/virtjdbc4_2-4.2.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
...The default terminal will act as a log window and after some time each one of the 2 experiments will conclude with the statistics report and export. As shown below, in red color, Virtuoso timedout during scanning while executing Query 24, but in the 180 sec time window managed to retrieve 3900 out of the 459523 results. Strabon on the other hand managed this spatial join query well within the 180 sec time window specified.
...
253377 [main] INFO GenericExprerimentResultsCollector - Export statistics in "/data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment"
253377 [main] INFO GenericExprerimentResultsCollector - Created non existing directory
253379 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-cold
253380 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-cold-long
253380 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/24-Synthetic_Join_Intersects_1_1-cold
253380 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/24-Synthetic_Join_Intersects_1_1-cold-long
253380 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-warm
253380 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-warm-long
253380 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/24-Synthetic_Join_Intersects_1_1-warm
253381 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/24-Synthetic_Join_Intersects_1_1-warm-long
253381 [main] INFO GenericExprerimentResultsCollector - Cache COLD
253381 [main] INFO GenericExprerimentResultsCollector - Query 0
253381 [main] INFO GenericExprerimentResultsCollector - Rep 0 <COMPLETED-NONE> 285400373 + 1986875274 = 2272275647 nsecs, 262144 results, 0 scan errors
253381 [main] INFO GenericExprerimentResultsCollector - Query 24
253381 [main] INFO GenericExprerimentResultsCollector - Rep 0 <SCANNING-TIMEDOUT> 5031666399 + 175243962593 = 180275628992 nsecs, 3900 results, 0 scan errors
253381 [main] INFO GenericExprerimentResultsCollector - Cache WARM
253381 [main] INFO GenericExprerimentResultsCollector - Query 0
253381 [main] INFO GenericExprerimentResultsCollector - Rep 0 <COMPLETED-NONE> 7376804 + 2070354419 = 2077731223 nsecs, 262144 results, 0 scan errors
253381 [main] INFO GenericExprerimentResultsCollector - Query 24
253381 [main] INFO GenericExprerimentResultsCollector - Rep 0 <SCANNING-TIMEDOUT> 5031666399 + 175243962593 = 180275628992 nsecs, 3900 results, 0 scan errors
253381 [main] INFO RunVirtuosoExperiment - End synthetic-synthetic
Start time = Sun Dec 17 10:09:19 UTC 2023
End time = Sun Dec 17 10:13:33 UTC 2023
...
85977 [main] INFO GenericExprerimentResultsCollector - Export statistics in "/data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment"
85977 [main] INFO GenericExprerimentResultsCollector - Created non existing directory
85980 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-cold
85980 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-cold-long
85980 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/24-Synthetic_Join_Intersects_1_1-cold
85980 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/24-Synthetic_Join_Intersects_1_1-cold-long
85980 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-warm
85980 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-warm-long
85981 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/24-Synthetic_Join_Intersects_1_1-warm
85981 [main] INFO GenericExprerimentResultsCollector - Statistiscs printed: /data/Results_Store/StrabonSUT/2023-12-03_StrabonSUT_Run_Synth/SyntheticWorkload/StrabonSUT-Experiment/24-Synthetic_Join_Intersects_1_1-warm-long
85981 [main] INFO GenericExprerimentResultsCollector - Cache COLD
85981 [main] INFO GenericExprerimentResultsCollector - Query 0
85981 [main] INFO GenericExprerimentResultsCollector - Rep 0 <COMPLETED-NONE> 1356973075 + 214905770 = 1571878845 nsecs, 262144 results, 0 scan errors
85981 [main] INFO GenericExprerimentResultsCollector - Query 24
85981 [main] INFO GenericExprerimentResultsCollector - Rep 0 <COMPLETED-NONE> 10168624855 + 519800276 = 10688425131 nsecs, 459523 results, 0 scan errors
85981 [main] INFO GenericExprerimentResultsCollector - Cache WARM
85981 [main] INFO GenericExprerimentResultsCollector - Query 0
85981 [main] INFO GenericExprerimentResultsCollector - Rep 0 <COMPLETED-NONE> 1157081594 + 68095670 = 1225177264 nsecs, 262144 results, 0 scan errors
85981 [main] INFO GenericExprerimentResultsCollector - Query 24
85981 [main] INFO GenericExprerimentResultsCollector - Rep 0 <COMPLETED-NONE> 8282911931 + 337278049 = 8620189980 nsecs, 459523 results, 0 scan errors
85981 [main] INFO RunStrabonExperiment - End synthetic-synthetic
Start time = Sun Dec 17 10:57:26 UTC 2023
End time = Sun Dec 17 10:58:52 UTC 2023
root@NUC8i7BEH:/data# tree -L 1 virtuoso-opensource/repos
virtuoso-opensource/repos
`-- synthetic
root@NUC8i7BEH:/data# su postgres
postgres@NUC8i7BEH:/data$ psql --list
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
------------------+--------------+----------+-------------+-------------+-----------------------
geographica3 | geographica3 | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
postgres | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
synthetic | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
template0 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template_postgis | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
postgres@NUC8i7BEH:/data$ exit
root@NUC8i7BEH:/data# ls -lsa geordfbench/VirtuosoSUT/scripts/CreateRepos/*.log
4 -rw-r--r-- 1 root root 3836 Dec 17 10:09 geordfbench/VirtuosoSUT/scripts/CreateRepos/logCreateRepos_Synthetic_Virtuoso.log
root@NUC8i7BEH:/data# ls -lsa geordfbench/StrabonSUT/scripts/PostgresUtils/CreateDatasetDBs/*.log
8 -rw-r--r-- 1 root root 8160 Dec 17 10:57 geordfbench/StrabonSUT/scripts/PostgresUtils/CreateDatasetDBs/logCreateRepo_Synth_Strabon.log
Experiment run logs may be quite long, therefore the user can click the links below to view the details of the execution.
root@NUC8i7BEH:/data# ls -lsa geordfbench/VirtuosoSUT/scripts/RunTests3/*.log
8 -rw-r--r-- 1 root root 22712 Dec 17 10:13 geordfbench/VirtuosoSUT/scripts/RunTests3/RunVirtuosoExp_Synth.log
root@NUC8i7BEH:/data# ls -lsa geordfbench/StrabonSUT/scripts/RunTests3/*.log
32 -rw-r--r-- 1 root root 28643 Dec 17 10:58 geordfbench/StrabonSUT/scripts/RunTests3/RunStrabonExp_Synth.log
Experiment results are stored by default in the file system. For this demonstration the base location for all results was /data/Results_Store. The output of the following command
has been modified to fully expand the last entry only, with the result of tree -L 4 Results_Store/VirtuosoSUT/ :
root@NUC8i7BEH:/data# tree -L 2 Results_Store/
Results_Store/
|-- StrabonSUT
| `-- 2023-12-03_StrabonSUT_Run_Synth
`-- VirtuosoSUT
`-- 2023-12-03_VirtuosoSUT_Run_Synth
`-- SyntheticWorkload
`-- VirtuosoSUT-Experiment
|-- 00-Synthetic_Selection_Intersects_1_1.0-cold
|-- 00-Synthetic_Selection_Intersects_1_1.0-cold-long
|-- 00-Synthetic_Selection_Intersects_1_1.0-warm
|-- 00-Synthetic_Selection_Intersects_1_1.0-warm-long
|-- 24-Synthetic_Join_Intersects_1_1-cold
|-- 24-Synthetic_Join_Intersects_1_1-cold-long
|-- 24-Synthetic_Join_Intersects_1_1-warm
`-- 24-Synthetic_Join_Intersects_1_1-warm-long
root@NUC8i7BEH:/data# more Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-cold
262144 2272275647
root@NUC8i7BEH:/data# more Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/00-Synthetic_Selection_Intersects_1_1.0-cold-long
262144 285400373 1986875274 2272275647
root@NUC8i7BEH:/data# more Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/24-Synthetic_Join_Intersects_1_1-cold
3900 180275628992
root@NUC8i7BEH:/data# more Results_Store/VirtuosoSUT/2023-12-03_VirtuosoSUT_Run_Synth/SyntheticWorkload/VirtuosoSUT-Experiment/24-Synthetic_Join_Intersects_1_1-cold-long
3900 5031666399 175243962593 18027562899
For each query and execution type (warm, cold) there are two files, a short and a long version. The long version has 4 columns (noOfResults, evaluationTime, scanTime, totalTime) and one row for each execution iteration performed. The short version has 2 columns (noOfResults, totalTime) and only one row which represents the average or median totalTime of the execution iterations from the long version. All times are in nano seconds. Please notice, that both experiments used the customized execution specification fast micro execution spec which dictates 1 repetition for COLD and 1 repetition for WARN caches.
Experiment results are also stored in a custom location, a PostgreSQL database. Each experiment details are recorded, with a unique ID, in a row of the EXPERIMENTS table. Each query execution iteration details are in a similar manner recorded in the QUERYEXECUTIONS table. A set of views can provide aggregation for the totalTime and calculation of the Average and Median value of totalTime for each query and execution type (warm, cold). All times are in milli seconds. In the figures below we can see the actual snapshots from the PgAdmin v4 interface with the experiment details on the left and the report results on the right side.
Below we present how we can transport the report data from Figure 2, to a LibreOffice Calc worksheet and create the corresponding charts. For simplicity purposes we will include the Virtuoso Failed executions which, depending on the benchmark researcher's point of view, can be excluded or included in the results discussion.
The above chart shows clearly that for the Synthetic workload, Strabon performed better than Virtuoso. In this chart we can also see, that for the non timed out queries the WARM cache times are lower than the COLD cache ones. This occurs even though it is known that clearing caches fail inside the container. However, both systems feature some kind of server component, Strabon features a PostgreSQL database server and Virtuoso an application server. Restarting the server components in such "server-based" RDF stores is part of the clear cache action employed before measuring a COLD run. Therefore in this situation we have a partial clear cache which explains the COLD to WARM cache execution time differences.
The modified fast micro execution spec, was designed with the intension to cause Virtuoso to timeout
while executing the spatial join query and thus provide us the opportunity to exhibit and study how GeoRDFBench Framework manages this situation. The following is an annonated
excerpt of the /data/geordfbench/VirtuosoSUT/scripts/RunTests3/RunVirtuosoExp_Synth.log log file:
38324 [main] INFO VirtuosoSUT - Starting Virtuoso server...
50329 [main] INFO VirtuosoSystem - Uninitialized VirtuosoRepository created with query timeout = 0
50329 [main] INFO VirtuosoSystem - Initialized VirtuosoRepository has query timeout = 180
50331 [main] INFO VirtuosoSUT - Starting QueryExecutor thread
50331 [main] INFO VirtuosoSUT - Waiting for QueryExecutor thread to finish
50331 [main] INFO VirtuosoSUT - Timeout progress step is 45000 msecs
50331 [Thread-2] INFO QueryRepResult - Transitioning (NOTSTARTED => STARTED)
50331 [Thread-2] INFO QueryRepResult - Transitioning (STARTED => PREPARING)
50332 [Thread-2] INFO QueryRepResult - Transitioning (PREPARING => EVALUATING)
55363 [Thread-2] INFO QueryRepResult - Transitioning (EVALUATING => EVALUATED)
55363 [Thread-2] INFO RDF4JbasedExecutor - s1 s2
55364 [Thread-2] INFO RDF4JbasedExecutor - ------------------------------------>
55364 [Thread-2] INFO QueryRepResult - Transitioning (EVALUATED => SCANNING)
55364 [Thread-2] INFO RDF4JbasedExecutor - http://geographica.di.uoa.gr/generator/landOwnership/85242/ http://geographica.di.uoa.gr/generator/state/9433/
55364 [Thread-2] INFO RDF4JbasedExecutor - http://geographica.di.uoa.gr/generator/landOwnership/85243/ http://geographica.di.uoa.gr/generator/state/9433/
55364 [Thread-2] INFO RDF4JbasedExecutor - http://geographica.di.uoa.gr/generator/landOwnership/85243/ http://geographica.di.uoa.gr/generator/state/9434/
95331 [main] INFO VirtuosoSUT - Percentage of expired timeout is 25.0 %
140332 [main] INFO VirtuosoSUT - Percentage of expired timeout is 50.0 %
185332 [main] INFO VirtuosoSUT - Percentage of expired timeout is 75.0 %
230333 [main] INFO VirtuosoSUT - Timeout expired! Sent interrupt to worker thread and waiting for it to join.
230608 [Thread-2] ERROR RDF4JbasedExecutor - RuntimeException - I wasn't done! Query prepared and evaluated. Scanning in progress ...3900 results, so far. Not big enough patience window defined... :(
230618 [main] INFO Experiment - |<== Executed query [24, Synthetic_Join_Intersects_1_1] (COLD, 0): 5031666399 + 175243962593 = 180275628992 nsecs, 3900 results, 0 scan errors - ACCURACY NOT DETERMINED
230618 [main] INFO VirtuosoSUT - Closing..
230618 [main] INFO VirtuosoSystem - Closing connection...
230619 [main] INFO VirtuosoSystem - Repository closed.
232628 [main] INFO VirtuosoSUT - Closed (caches not cleared)
232628 [main] INFO VirtuosoSUT - Stopping Virtuoso server...
The above log excerpt, focuses on the COLD execution of Query 24 Synthetic_Join_Intersects_1_1 by Virtuoso. Under GeoRDFBench a query is run as described below:
The more interested user, can look at the simple Bash script, /data/startUpScript.sh, which launched the experiments. The actions taken by this script are: