Technical Overview
Integration of World-Popular Cloud Storage with Gfarm
Nextcloud is used worldwide as an open-source file synchronization & sharing platform. Nextcloud support (Nextcloud-gfarm) is a technology that leverages the powerful features of Nextcloud to integrate the Gfarm File System as Nextcloud's external storage.
Nextcloud-gfarm maintains the capacity & performance of Gfarm's large-scale distributed file system while providing users with Nextcloud's familiar interfaces (web, desktop client, mobile apps, etc.). Authentication uses OAuth/OIDC authentication, a modern security standard. By configuring the JWT server with username and passphrase, access tokens are automatically refreshed, enabling secure and continuous access to Gfarm.
Features & Strengths
Large-Scale Data Management with a Familiar Interface
1Large-Scale Data Management with a Familiar Interface
Gfarm's large-capacity storage can be easily operated through Nextcloud's web interface. This allows users to manage Gfarm data using Nextcloud's rich features such as file sharing, version control, tagging, & favorites.
2Secure & Continuous Access
By utilizing the OAuth/OIDC authentication mechanism to automatically refresh access tokens, users can continue using Gfarm data without worrying about authentication expiration while maintaining security.
3High Reliability
Nextcloud-gfarm has a proven track record of actual operational use in HPC environments such as the Pegasus supercomputer at the University of Tsukuba, demonstrating its robustness & reliability.
4Easy Deployment
Published on GitHub, you can easily launch a Nextcloud container with Gfarm access by following the Quick Start procedure & creating/configuring the configuration files.
Usage Examples
Cloudifying Research Environments
- Collaborative Work on Research Data
-
Securely share large-capacity experimental data stored in Gfarm with research collaborators using Nextcloud's sharing features, & proceed with collaborative editing & analysis.
- Personal Cloud in HPC Environments
-
Use the supercomputer's file system as a personal cloud accessible via the web from your PC or tablet to organize & review data.
- Integration with External Services
-
Build various services (document editing, calendar integration, etc.) utilizing data on Gfarm through Nextcloud's rich app integration capabilities.
Target Needs & Users
- Organizations with Nextcloud Already Deployed
-
Organizations already using Nextcloud that want to significantly expand their backend storage capacity and performance with Gfarm.
- Supercomputer Users
-
Researchers who want to manage computational result data through an intuitive, familiar cloud interface rather than specialized command operations.
- System Integrators
-
System administrators who want to securely integrate large-scale data infrastructure (Gfarm) with a user-friendly frontend (Nextcloud) using proven methods.
Other Technology & Development
- OAuth/OIDC Authentication
- This mechanism enables more secure access to the Gfarm file system using a new login method.
- Gfptar
- Parallel transfers large numbers of input entries (files) while automatically consolidating them into multiple archive files within the output directory.
- Gfarm HTTP Gateway
- A mechanism enabling access through the HTTP protocol, widely used in web applications.
