How to mount EODATA on Windows virtual machine on WEkEO Elasticity hosting

WEkEO Elasticity cloud contains EODATA, which is a repository of Earth observation data. In this article you will learn how to access it from a virtual machine running Windows in the cloud.

Machines created on or after the 20th of December 2023 should have this process completed automatically within a few minutes after the first successful login (if they are connected to your project network and the eodata_ network). If your VM does not have that software, you can

  • use one of the methods listed in the article to configure EODATA or

  • create a totally new instance using a sufficiently new image which is able to mount the EODATA repository automatically.

What We Are Going To Cover

  • Accessing EODATA on virtual machines with automatic EODATA mounting

  • Two ways of accessing EODATA on virtual machines without automatic EODATA mounting

    1. Using free and open source tools: Rclone, WinFSP and NSSM

    2. Using the free version of Air Live Drive

Prerequisites

No. 1 Account

You need a WEkEO Elasticity hosting account with access to the Horizon interface: https://horizon.cloudferro.com.

No. 2 Virtual machine with Windows

If you don’t already have a virtual machine, use article How to create new Linux VM in OpenStack Dashboard Horizon on WEkEO Elasticity as the basic blueprint for creation of an instance.

Apply the following changes to the procedure in that article:

For Source, use a Windows image.

For Networks, be sure to add two networks, one starting with cloud_ and the other starting with eodata_.

If you do not want to access your instance outside of the Horizon dashboard, you may omit the following from the workflow described in that article:

  • creation of a floating IP,

  • using the allow_ping_ssh_icmp_rdp security group as well as

  • the instructions regarding the SSH connection.

The instructions regarding SSH connection from that article do not apply to Windows VMs. Virtual machines with Windows are typically controlled using RDP. If you choose this method, you might want to consider using bastion host forwarding to increase the security of your connection: Connecting to a Windows VM via RDP through a Linux bastion host port forwarding on WEkEO Elasticity

Accessing EODATA on virtual machines with automatic EODATA configuration

Make sure that your virtual machine should be connected to your project network and to the eodata_ network.

Enter the Administrator system account on your virtual machine.

Go to This PC window. The EODATA repository should be mounted there under drive letter Y in the Devices and drives section:

../_images/winvirtmach-mount-eodata-windows-1_creodias.png

Note

If you do not see the repository there, wait up to a minute until the mounting process is completed.

Double-click it. You should now see the contents of EODATA repository:

../_images/winvirtmach-mount-eodata-windows-2_creodias.png

You might recognize folder names such as Landsat-5 or Sentinel-2 as the names of actual satellites from which the images are gathered.

You can now copy the resources to the hard drive of your VM or open them using your processing software. If you are using the latter option, make sure that your software does not try to save its results to the EODATA repository since you do not have write permissions there.

Note

The rest of the article covers virtual machines created using images without automatic mounting.

Accessing EODATA on virtual machines without automatic EODATA mounting

We will cover the following methods of configuring access to the EODATA repository:

  • Using open source software. In this method, tools such as Rclone, WinFSP and NSSM will be deployed.

  • Using Air Live Drive. Here, the free version of this proprietary piece of software will be utilized.

The free version of Air Live Drive (used in Method 2) should be sufficient if you only wish to access the EODATA repository on your virtual machine and not other object storage. If you, however, want to access for example private object storage containers on WEkEO Elasticity WAW3-1 alongside the EODATA repository, you should use Method 1.

Method 1: Accessing EODATA with open source software

Software tools used in this method: Rclone, WinFSP, and NSSM

Rclone has multiple functions such as managing files in cloud storage and syncing between file systems. In this article, you will use its rclone mount command to mount object storage on your Windows VM.

WinFSP enables accessing custom file systems on Microsoft Windows. In this workflow, it will allow Rclone to mount the S3 storage.

NSSM is a service manager. Here, it will be used for configuring automatic mounting of object storage. You will run it from the command line.

Preparing the environment

Access the Administrator account on your virtual machine. Open PowerShell and execute the following command to create a directory in which Rclone will be stored and navigate to it:

mkdir C:\rclone; cd C:\rclone

Without closing PowerShell, open a web browser on your virtual machine. Navigate to the following website: https://rclone.org/downloads. It should contain the table with links to its different versions:

../_images/winvirtmach-mount-eodata-windows-open-03_creodias.png

Download the version of Rclone for the Intel/AMD - 64 Bit platform (for Windows operating system).

A zip archive should be downloaded. In the Microsoft Edge browser it should look like this:

../_images/winvirtmach-mount-eodata-windows-open-04_creodias.png

Enter that zip archive, it should contain one directory. Double-click to enter it. Its content should look like this:

../_images/winvirtmach-mount-eodata-windows-open-05_creodias.png

Copy the content of that directory to the C:\rclone folder which you previously created using PowerShell.

Return to your Internet browser. Navigate to the following website: https://winfsp.dev/rel/. Click Download WinFsp Installer:

../_images/winvirtmach-mount-eodata-windows-open-06_creodias.png

Run the downloaded installer. The installation process will be similar to installation of other Windows programs. Its step called Custom Setup, however, will require you to make a choice:

../_images/winvirtmach-mount-eodata-windows-open-07_creodias.png

Leave the default values intact to install only the Core section and click Next.

Once you have installed it, open the Internet browser on your Windows virtual machine. Navigate to https://nssm.cc/download

Click the link in the Latest relase section:

../_images/winvirtmach-mount-eodata-windows-open-10_creodias.png

A zip file should be downloaded. It should contain one folder. The content of that folder should look like this:

../_images/winvirtmach-mount-eodata-windows-open-11_creodias.png

Navigate to win64 folder. It should have one executable file called nssm:

../_images/winvirtmach-mount-eodata-windows-open-12_creodias.png

Copy that file to C:\rclone

Mounting EODATA

Create the following directory and navigate to it using the Windows file manager:

C:\Users\Administrator\.config\rclone

Create a text file called rclone.conf there. Make sure that the extension of this file is .conf and not .txt. Open it using Notepad or other plain text editor like Notepad++.

Each section in that file responsible for mounting of object storage starts with its name written in square brackets. Below it are parameters describing that connection to the object storage container, including the key pair. Add the section below to that file. It contains information used to connect to the EODATA repository.

In the configuration found below, replace CLOUDFERRO and PUBLIC with the access key and the secret key you are using for mounting EODATA, respectively. The method of obtaining those credentials varies depending on the cloud you are using. See How to get credentials used for accessing EODATA on a cloud VM on WEkEO Elasticity for more information.

[EODATA]
type = s3
provider = Other
access_key_id = CLOUDFERRO
secret_access_key = PUBLIC
endpoint = http://data.cloudferro.com

Note

Make sure to use the right prefix - http:// or https:// for the endpoint in your configuration.

Return to PowerShell. If you aren’t in the C:\rclone directory there, navigate to it using the following command:

cd C:\rclone

Now, execute the following command to test the connection to the object storage container:

.\rclone.exe lsd EODATA:DIAS

You should see output similar to this:

../_images/winvirtmach-mount-eodata-windows-open-ps-02_creodias.png

To test whether mounting works, execute the command below. If you don’t want the EODATA repository mounted under the Y drive letter, feel free to modify that command accordingly.

.\rclone.exe mount --read-only EODATA:DIAS Y:

You should get the following message:

The service rclone has been started.

Navigate to the This PC window. The EODATA repository should be mounted there:

../_images/winvirtmach-mount-eodata-windows-open-08_creodias.png

Its content should look like this:

../_images/winvirtmach-mount-eodata-windows-open-09_creodias.png

Press CTRL+C when the PowerShell window is focused to stop the test mount. You should get the following output:

The service rclone has been stopped.

The EODATA repository should no longer be visible in This PC window.

If pressing CTRL+C does not stop the test mount, make sure that the PowerShell window is focused by left-clicking it. Press a letter on your keyboard, for example A, and try pressing CTRL+C again.

Do not close PowerShell yet.

Configuring automatic mounting of EODATA

You have successfully mounted the EODATA repository on your Windows virtual machine. However, after rebooting your virtual machine, you will need to mount it again using Rclone. To prevent this, you can configure its automatic mounting.

If the previous instance of PowerShell is still running, close it. This should cause the EODATA repository to disappear from your This PC window. If that has not happened, refresh that window (right-click and choose Refresh).

Return to PowerShell and execute the following command:

.\nssm.exe install

The following window should appear:

../_images/winvirtmach-mount-eodata-windows-open-nssm-01_creodias.png

Click the … button next to the Path: text field.

Choose the location of Rclone. If you followed this tutorial, this location is as follows:

C:\rclone\rclone.exe

In the Arguments text field enter the following code. Replace E: with the drive letter under which you wish to mount it.

mount --read-only EODATA:DIAS Y:

Warning

Make sure that you do not have anything mounted under the drive letter you choose here.

In the text field Service name: enter EODATA.

Navigate to the Log on tab. Choose This account:. In the text field next to that option enter Administrator. Enter the password to your Administrator account in the Password: and Confirm: text fields.

Click Install service.

Return to PowerShell and execute the following command to start the service you just created:

.\nssm.exe start EODATA

You should get the following message:

EODATA: START: The operation completed successfully.

Now the EODATA repository should always be mounted, even after rebooting your virtual machine. To test it, you can reboot your machine and check whether the EODATA repository is still in the This PC window.

You can now copy the resources to the hard drive of your VM or open them using your processing software. If you are using the latter option, make sure that your software does not try to save its results to the EODATA repository since you do not have write permissions there.

Method 2: Accessing EODATA with Air Live Drive

Air Live Drive is a proprietary piece of software. Its free version should be sufficient for accessing the EODATA repository. In comparison to the previous method, its configuration is performed using the graphical user interface.

Download Air Live Drive

Log in to the Administrator account of your virtual machine.

Navigate to https://www.airlivedrive.com/en/download/ using the Internet browser.

Click DOWNLOAD:

../_images/winvirtmach-mount-eodata-windows-ald-01_creodias.png

Install Air Live Drive

Execute the downloaded installer and install the program.

Mount EODATA

After installing and running the program, its window should appear:

../_images/winvirtmach-mount-eodata-windows-ald-02_creodias.png

Click Add Cloud Disk. You should see the following window:

../_images/winvirtmach-mount-eodata-windows-ald-03_creodias.png

Click S3.

The form used to enter the access data should appear:

../_images/winvirtmach-mount-eodata-windows-ald-04_creodias.png

In the text field REST endpoint type the following:

http://data.cloudferro.com

Obtain the credentials you are using for accessing the EODATA repository. The procedure varies depending on the cloud you are on. See How to get credentials used for accessing EODATA on a cloud VM on WEkEO Elasticity for more information.

Enter the appropriate access and secret key in the Access Key ID and Secret Key ID text fields, respectively.

Select the check box Connecto to a specific bucket.

In the text field Bucket: type DIAS.

Your form should now look like this:

../_images/winvirtmach-mount-eodata-windows-ald-05_creodias.png

Click OK.

The EODATA repository should now be visible in the Air Live Drive window:

../_images/winvirtmach-mount-eodata-windows-ald-06_creodias.png

Click the gear icon to the right of the Connect button. The following window should appear:

../_images/winvirtmach-mount-eodata-windows-ald-07_creodias.png

Select Mount as read only.

Optionally, you can choose Connect at startup if you want the mounting process to happen automatically after running Air Live Drive.

Click Close.

Before mounting, you can choose the drive letter using the drop-down menu.

Click Connect. You should now see the window with the content of the EODATA repository:

../_images/winvirtmach-mount-eodata-windows-ald-08_creodias.png

This repository should also be visible in the This PC window:

../_images/winvirtmach-mount-eodata-windows-ald-09_creodias.png

Note

If you want Air Live Drive to start automatically after Windows has started, go to its Options using the gear icon in the lower-left section of the window and check Run Air Live Drive at login.

You can now copy the resources to the hard drive of your VM or open them using your processing software. If you are using the latter option, make sure that your software does not try to save its results to the EODATA repository since you do not have write permissions there.

Stopping automatic mounting of the EODATA repository

Depending on which method of configuring access to the EODATA repository you chose, the procedure will vary.

Stop automatic mounting in case of Air Live Drive

Open Air Live Drive. As before, you should see the EODATA repository mounted:

../_images/winvirtmach-mount-eodata-windows-ald-stop-01_creodias.png

Now, depending on whether you wish to access the EODATA repository using this method later, follow one of these workflows:

Stop automatic mounting but keep connection data

In the section of Air Live Drive containing connection data to the EODATA repository click Disconnect. You should get the following question:

../_images/winvirtmach-mount-eodata-windows-ald-stop-02_creodias.png

Make sure that none of your applications tries to access the EODATA repository using Air Live Drive and click Yes.

Once the drive is no longer visible in This PC window and the button which previously said Disconnect now says Connect, click the gear icon gear-icon in the same section. You should get the following window:

../_images/winvirtmach-mount-eodata-windows-ald-stop-03_creodias.png

Uncheck Connect at startup and click Close.

The EODATA repository should no longer be automatically mounted.

Remove connection to the EODATA repository

In the section of Air Live Drive containing connection data to the EODATA repository click the trash can icon: trash-can-icon.

You should get the following question:

../_images/winvirtmach-mount-eodata-windows-ald-stop-04_creodias.png

Make sure that none of your applications tries to access the EODATA repository using Air Live Drive and click Yes.

Wait up to a minute until the connection disappears from Air Live Drive:

../_images/winvirtmach-mount-eodata-windows-ald-stop-05_creodias.png

Optionally, uninstall Air Live Drive using the Control Panel.

Stop automatic mounting in case of Virtual machine with preconfigured EODATA mounting

Navigate to C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup using the Windows file manager.

Note

You might need to enable viewing of hidden files and folders in Windows.

You should find the batch file called mount_eodata.bat there:

../_images/winvirtmach-mount-eodata-windows-open-stop-preconfigured-01_creodias.png

This batch file executes the PowerShell script called mount_eodata.ps1 found in C:\rclone. The script mount_eodata.ps1 creates an Rclone configuration file if it doesn’t exist and mounts the EODATA repository.

Delete the mount_eodata.bat file from the C:\ProgramData\MicrosoftStart Menu\Programs\Startup folder. In case you later need to restore it, you can find its backup in the mount_eodata folder on your Desktop. That folder also contains backup of the mount_eodata.ps1 PowerShell script found in C:\rclone.

Reboot your VM. The EODATA repository should no longer be automatically mounted.

If you want to delete connection data, open the C:\Users\Administrator\.config\rclone.conf file using Notepad or other plain text editor and remove the lines responsible for mounting of EODATA. They will look similarly to this (the login credentials might be different):

[EODATA]
type = s3
provider = Other
access_key_id = access
secret_access_key = secret
endpoint = http://data.cloudferro.com

Stop automatic mounting in case of open source software configured manually

Open PowerShell.

Navigate to the C:\rclone directory by executing the following command:

cd C:\rclone

Check the status of the service used for mounting of EODATA:

.\nssm.exe status EODATA

You should get the following output:

SERVICE_RUNNING

To stop the automatic mounting of your container, execute the command below. Replace mounting-service as previously.

.\nssm.exe stop EODATA

Delete the service by executing the command below. Replace mounting-service as previously.

.\nssm.exe remove EODATA confirm

You should now get the output similar to this:

Service "EODATA" removed successfully!

If you want to delete connection data, open the C:\Users\Administrator\.config\rclone.conf file using Notepad or other plain text editor and remove the lines responsible for mounting of EODATA. They will look similarly to this (the credentials might be different):

[EODATA]
type = s3
provider = Other
access_key_id = CLOUDFERRO
secret_access_key = PUBLIC
endpoint = http://data.cloudferro.com

What To Do Next

If you want to access EODATA on your Linux virtual machine, follow this article instead:

How to mount eodata using s3fs in Linux on WEkEO Elasticity.

There are other options, such as:

If you followed Method 1 in this article or have a VM with automatic mounting of the EODATA repository, you can use the open source software mentioned in this article to access private object storage containers from WEkEO Elasticity cloud. Check this text to learn more: How To Mount Object Storage Container as File System on Windows VM on WEkEO Elasticity.