Technical Overview
Making Gfarm File System Accessible Through the Power of the Web
Gfarm HTTP Gateway is a mechanism that enables access to Gfarm, a large-scale distributed file system, through the HTTP protocol widely used in web applications. At the core of this technology are OAuth/OIDC authentication & JWT (JSON Web Token). Traditionally, accessing Gfarm required dedicated client software, but through this Gateway, users can access Gfarm simply by adding a JWT to the Authorization header of HTTP requests.
This makes it possible to perform various Gfarm operations (file upload, download, information retrieval, etc.) through standard HTTP clients like curl & REST APIs. Furthermore, Gfarm Web App, which leverages this Gateway, has been developed. This is an application that enables the web browser itself to function as a Gfarm client, providing visual & intuitive operations.
Features & Strengths
Easy Access & High Security
1Easy Integration with Existing Systems
Even in environments where Gfarm client software is not installed, access is possible from anywhere that HTTP protocol can be used. This significantly facilitates Gfarm integration from web applications & scripts.
2Intuitive Operations via Web Browser
Using Gfarm Web App, everyday file operations such as displaying directory lists, downloading/uploading files, & renaming are possible from web browsers on PCs or smartphones.
3Balancing Security & Convenience
Upon login, encrypted refresh tokens for automatically updating access tokens are stored. The encryption key is stored in the web browser's cookies & is not retained on the Gfarm Web App side, enabling secure storage of refresh tokens. During file operations, access tokens are automatically refreshed through the Gfarm HTTP Gateway.
Usage Examples
Data Access from Anywhere
- Simple File Downloads
-
Using the curl command, you can directly download large-capacity files from Gfarm without complex configuration.
- Research Data Visualization & Sharing
-
Directory structures & file information (size, owner, update time, etc.) can be easily checked through a web browser, making data sharing & verification work within research teams more efficient.
- Smartphone Usage
-
Since the web browser serves as the client, users can access Gfarm from smartphones & manage or view data even while on the go.
Target Needs & Users
- Web System Developers
-
Those who want to provide an easy-to-use web interface on the frontend while having Gfarm's vast data as the backend.
- Remote Working Researchers
-
Those who want to securely access large-scale research data from remote locations without installing dedicated client software.
- System Administrators
-
Those who want to facilitate data management using standard HTTP tools and integration with automation scripts.
Other Technology & Development
- OAuth/OIDC Authentication
- This mechanism enables more secure access to the Gfarm file system using a new login method.
- Gfptar
- Parallel transfers large numbers of input entries (files) while automatically consolidating them into multiple archive files within the output directory.
- Nextcloud Support
- By accessing shared storage directly, users can more easily utilize data on Gfarm.
