| Version 15 (modified by mtmoore, 2 years ago) |
|---|
OrangeFS GSoC2011 Project Page
Idea List
OrangeFS is written, except for a small group of utilities, in C. A successful application requires experience with C.
The OrangeFS server and primary client are written for Linux. A successful application requires experience using Linux.
'Experience' does not need to be formal like a job or course work. A description of experience or reference to previous work using C or Linux is acceptable. The key is to communicate that your experience and skill set are adequate for the project you are proposing.
File System Check
Background
As with most file systems, OrangeFS has a utility to repair the file system. The current fsck only operates when the file system is offline (in administrative mode). Also, as new functionality has been added appropriate checks to ensure consistency may not have been added.
Idea
The goal for this project is to evaluate the current tool for correctness of existing checks, identify additional checks that should be performed to ensure file sytsem consistency, and identify repair operations that can successfully be performed while the file system is online (remember, OrangeFS does not have locking). The proposal should meet these goals:
- Evaluate the current fsck tool; it may be broken, deficient, or mostly adequate.
- Implement needed changes to the offline fsck process to ensure a correct file system. This may correcting existing or adding new checks
- Add online fsck repair options
Asynchronous File Delete
Operation Log
Background
Determining what operations a server is currently handling, or has been servicing, is currently only possible by enabling very verbose logging. Using textual logging (even just the operation information) is not acceptable due to the performance cost. Also, enabling verbose debugging generates too much information to allow a user to readily skim out what requests the server is processing.
Idea
The goal of this idea is to provide a method of inspecting requests received by a specific server and making them available for review by a user (administrator). One idea, provided to explain the idea not to indicate it's the best or only way, is to provide an in-memory buffer to store requests. A client can then send a command to dump the requests into a file (or send back over the network). Next, a utility can be used to parse and display results. A proposal may offer different solutions to the problem but the general goals are:
- minimal performance impact when operation logging is enabled
- the ability to dynamically enable and disable operation logging
- a mechanism to dump stored operations to a persistent store or to a client to persistently store
- a new, or extension of an existing utility, to display the operation history in a user consumable format.
Persistent Client Cache
Background
Currently, when applications use the shared library to make requests (using pvfs2-* commands, MPI) the cache of each process is lost when the process terminates. In cases where shell scripts are used this leads to almost no benefits from caching. We are currently developing a LD_LIBRARY_PRELOAD mechanism that would allow for general use of OrangeFS filesystems without relying on the kernel module. However, this interface will also have the same cache issues since it utilizes the shared library for file operations.
Idea
There may be several ideas on how to resolve this issue. One idea is to utilize a shared memory area, managed by a background process, the shared library can store and retrieve attribute, data, and name cache information from. The shared memory area would be persistent between an individual user's command invocations. In effect, this cache would operate the same as the kernel module's cache. The implementations of the cache for each type (attribute and name) exist except for data.
Proposals can include alternative solutions to the problem outlined in the background.
Server Performance Monitoring
background
Currently, performance metrics are available through an OrangeFS client operation to the servers. The metrics returned are a large array of counters collected at a fixed time frame.
Idea
The goal of this project is to export OrangeFS performance metrics via SNMP. The project could include a redesign and implementation of a new performance monitoring infrastructure or utilize the current performance monitoring interface. In either case, the information should be exposed through SNMP (net-snmp) for consumption by monitoring tools that talk SNMP. If a new infrastructure is proposed it should not expose less information or prevent client tools from accessing performance information. If due to the scope of changes necessary to support SNMP polling, the SNMP module is not produced, the work should provide an interface the module could use.
pNFS
Background
A file layout driver was written for PVFS2 to support pNFS. Currently, those changes are available as patches to OrangeFS. However, the patch set has not been maintained as changes to the OrangeFS code have been incorporated.
Idea
This project will include reviewing, possibly modifying, and committing existing pNFS patches to the development branch of OrangeFS. Additionally, necessary configure and build options to support pNFS support should be made. A successful proposal will also include testing. After incorporating the changes the implementation must be tested against a current NFS server (one that supports pNFS). If the changes necessary to update and test the pNFS patches are minor the project can also include performance testing (systems will be made available) with comparison to native OrangeFS access. Also, appropriate documentation to enable pNFS support should be provided.
