IBM Aspera with HSTS

The following instructions are intended to provide a seamless and secure deployment of IBM Aspera with HSTS in a Windows or Linux (RedHat) environment, ensuring optimized performance for moving data to Lyve Cloud Object Storage. Adjustments may be made as needed based on specific organization policies and infrastructure.

Pre-Deployment

Network Configuration

Ports

Ensure the following are accessible:

• Aspera TCP/UDP Port: 33001 (default for data transfer)
• SSH Port: 22 (keep enabled initially to avoid accidental lockouts)

Firewall Rules

Permit inbound and outbound traffic on necessary ports. Verify the port status:

# netstat -na |grep 33001

SSH Security

Configure SSH Daemon

Open the sshd_config file and set:

• AllowTcpForwarding no
• AllowAgentForwarding no
• PubkeyAuthentication yes
• PasswordAuthentication yes
• port 33001
• port 22

Restart SSH Service

Apply changes:

# systemctl restart sshd.service

Verify that port 33001 is now listening:

# netstat -na |grep 33001
tcp        0      0 0.0.0.0:33001               0.0.0.0:*                           LISTEN
tcp        0      0 10.0.10.242:33001       125.20.120.90:52332      ESTABLISHED
tcp        0      0 10.0.10.242:33001       125.20.120.90:57615      ESTABLISHED
tcp        0      0 10.0.10.242:33001       14.194.8.182:62305        ESTABLISHED
tcp6      0      0 :::33001               

HSTS Installation and Verification

Installation

Install HSTS on RedHat using:

# rpm -Uvh ibm-aspera-hsts-<version>-linux-64-release.rpm

To verify the installation:

• HSTS Version: # ascp -A
• License Validation: # cat /etc/aspera-license

User and Environment Setup

User Creation

Create and configure a default user (svcAspera):

# useradd svcAspera
# su svcAspera
# sudo chsh -s /bin/aspshell svcAspera

SSH Key Setup

Establish user SSH credentials: 

# sudo mkdir /home/<username>/.ssh
# sudo chmod 700 /home/<username>/.ssh
# sudo touch /home/<username>/.ssh/authorized_keys
# sudo chmod 600 /home/<username>/.ssh/authorized_keys

Verify ssh and network ports are open for Aspera and HSTS:

# systemctl restart sshd.service
# netstat –ha |grep 33001
# default TCP/UDP ports Aspera uses.

Configuring Transfer Parameters

Global Transfer Settings

To enforce security policies on data transfers, configure the following: 

# sudo asconfigurator -x "set_node_data;authorization_transfer_in_value,deny"
# sudo asconfigurator -x "set_node_data;authorization_transfer_out_value,deny" 

User-specific Parameters

Set specific transfer permissions and bandwidth for user=svcAspera:

# sudo asconfigurator -x "set_user_data;user_name,svcAspera;authorization_transfer_in_value,allow"
# sudo asconfigurator -x "set_user_data;user_name,svcAspera;transfer_in_bandwidth_flow_target_rate_default,96000"

# sudo asconfigurator -x "set_node_data;authorization_transfer_in_value,deny"
# sudo asconfigurator -x "set_node_data;authorization_transfer_out_value,deny"

# sudo asconfigurator -x "set_user_data;user_name,test_aspera;authorization_transfer_in_value,allow"
user_name: svcAspera
# sudo asconfigurator -x "set_user_data;user_name,test_aspera;authorization_transfer_out_value,allow"
success
user_name: svcAspera
# sudo asconfigurator -x "set_user_data;user_name,test_aspera;absolute,s3:// s3.us-west-1.sv15.lyve.seagate.com/aspera-test-bucket"
success
user_name: svcAspera
# sudo asconfigurator -x "set_user_data;user_name,test_aspera;transfer_in_bandwidth_flow_target_rate_cap,unlimited"
success
user_name: svcAspera
# sudo asconfigurator -x "set_user_data;user_name,test_aspera;transfer_out_bandwidth_flow_target_rate_cap,unlimited"
success
user_name: svcAspera
# sudo asconfigurator -x "set_user_data;user_name,test_aspera;transfer_in_bandwidth_flow_target_rate_default,96000"
success
user_name: svcAspera
# sudo asconfigurator -x "set_user_data;user_name,test_aspera;transfer_out_bandwidth_flow_target_rate_default,96000"
success
user_name: svcAspera

Reference

ascp is IBM Aspera’s version of the UNIX tool – scp, and async is IBM Aspera’s version of the UNIX tool rsync. Many of the arguments are similar but the internals of how the files are transferred are very different. More information about the tools can be found at:

• ascp: https://www.ibm.com/docs/en/ahts/4.4.x?topic=atfcl-ascp-command-reference
• async: https://www.ibm.com/docs/en/ahts/4.4.x?topic=ra-async-command-reference

Enable Trapd for Object Storage (Optional)

Trapd is the Aspera service that enables it to write to the object storage (including Hadoop Distributed File System (HDFS). Trapd is supported in HSTS for Linux 64-bit and Windows 64-bit. HSTS should be in close proximity to the object storage to minimize latency.

Note—Trapd is disabled by default in HSTS.

• Enable Trapd: # /opt/aspera/bin/astrap-config.sh enable
• Disable Trapd (if switching to another storage configuration): # /opt/aspera/bin/astrap-config.sh disable

When switching from virtual node to pvcl_cloud or use another option, link the pvcl_cloud kernel to ensure Trapd is indeed disable.

#/opt/aspera/bin/astrap-config.sh disable
#ln -s /opt/aspera/lib/pvcl/libpvcl_cloud.so /opt/aspera/lib/libpvcl_cloud.so

Use a text editor to open the s3.properties file (/opt/aspera/etc/trapd/s3.properties) and make the following changes:

s3service.https-only=true
s3service.s3-endpoint=s3.sv15.lyve.seagate.com       ## for example
s3service.s3-endpoint-https-port=443
s3service.disable-dns-buckets=true
s3service.use-path-style-url=true 

Restart the trapd service: 

# sudo systemctl restart asperatrapd 

S3 Configuration for Object Storage

Edit the s3.properties file (/opt/aspera/etc/trapd/s3.properties) to configure your S3 endpoint for secure, path-style URL usage:

s3service.https-only=true
s3service.s3-endpoint=<s3-endpoint>
s3service.s3-endpoint-https-port=443
s3service.use-path-style-url=true

where <s3-endpoint> is your S3 endpoint URL.

Restart trapd to apply these settings:

# sudo systemctl restart asperatrapd

IBM Aspera Desktop Client Setup (Windows)

Installation

1. Install the IBM Aspera Desktop Client on Windows using the provided installer, for example, IBMAsperaDesktopClient-ML-4.4.3.891-win-v143-64-release.exe. Proceed through the setup wizard steps.
2. When the installation is complete, launch Aspera Desktop Client on Windows.
3. Configure your S3 Connection parameters:

When using the default user:

The user credential and S3 endpoint access URL can be configured on either HSTS server side or alternatively on the Windows client side.

To configure the user credential on the Windows client side, the user credentials and S3 endpoint URL information need to be commented out in aspera.conf.

For configuration on the HSTS, the user credential and S3 access endpoint information are added in aspera.conf, following this format:

s3://<access-key>:<secret-key>@<s3-endpoint>

where:

• <access-key> is your access key.
• <secret-key> is your secret key.
• <s3-endpoint> is the appropriate Lyve Cloud S3 endpoint URL, for example, us-east-1.

1. Ensure the TCP/UDP ports are open.





2. Set up initial transfer speed parameters.





3. Test the setup from an Aspera Windows Client.
4. Launch Connection Manager.
5. Send files to and receive files from S3 cloud storage.