2009 Ad Hoc Developers Meeting Minutes

Meeting Minutes from the 2009 Ad Hoc Developers Meeting

Date: October 19, 2009
Place: Chapel Hill, NC

Meeting facilitated by Zac Adelman (UNC). Meeting started with introductions, who is in attendance, what are you doing, and what interests you in being involved in the ad hoc developers meeting. Questions that we would like to discuss:

• what are the perspectives in the community about development on the CMAS-supported software?
• does there need to be a development branch of the CMAQ code in addition to the official release version?
• what's the best way to facilitate development in the community

Initial discussions focused on code sharing and setting up an informal way for passing codes between community members. A code sharing forum, like Sourceforge, was suggested to house a version of the model to be used for testing new modules. This system should be a repository that community members could be granted permission to contribute code to for sharing with others.

There was some expressed discontent with CVS and discussion about using other version control systems like Subversion or Tract.

Some fundamental questions were raised:

• Does community modeling encourage innovation or stifle it?
• Is CMAQ developing faster or slower than similar models?
• If CMAQ is developing at the required speed, is it moving in the right direction?

The current state of CMAQ as a development/teaching platform was questioned. The modularity and clarity of the CMAQ codes was called into question for making it a good development platform.

Other examples in the environmental modeling community were discussed in terms of examples of whether community modeling an work. The point was raised that WRF does not provide a good model for how peer production can work effectively. It's a good repository of science but apparently a bad architecture. GEOS-CHEM was discussed as a model for how community modeling can work, although there were a lot of hurdles to get to a working system for that community.

GEOS-CHEM uses a steering committee to decide development and release directions for different versions of the model. Different members of the GEOS-CHEM community manage different pieces of the code base. The codes are distributed as tar balls from an ftp site. Developers go to the ftp site to get beta codes and releases. Updates and bug fixes to the code are distributed through the GEOS-CHEM wiki. It is expected in that community that any code changes made to the model will come back to the person responsible for that piece of the code base. GEOS-CHEM currently has 6-8 steering committees managing difference pieces of the code base. Emails sent to the GEOS-CHEM listserv are posted to the wiki. All communications about GEOS-CHEM are accessible through the wiki.

The existence of the CMAS developers meeting demonstrates that community modeling is working. The problem that we face is that CMAQ is moving in so many different directions that it's difficult to track and control

Some discussion focused on the direction that the meeting attendees would like CMAS to go in pursuing a development platform:

• The platform should be elegant and simple in structure
• The codes should avoid being deeply layered to promote understanding of how the calculations flow through the model
• Basic coding style should be implemented, including more strict rules about how in-line comments are handled
• Better documentation is needed, particularly a way to handle outside contributions

The prospect of a development branch of CMAQ was appealing to some meeting attendees. The idea is that there would be a main operational branch of the model that was maintained by EPA and a development/research branch maintained by the community. Modules from the development branch could be incorporated into the operational branch as appropriate. The development branch needed to be fairly simple and scalable. MAQSIP was proposed as a candidate for the development branch. Dr. Coats of Baron AMS suggested that he had a simple version of CMAQ that could possibly be used for development. The development version should use a hybrid OpenMP/MPI implementation for multi-processing environments.

The idea of a development version of a regulatory AQM is problematic because of the possibility that unofficial versions of the model will be used for regulatory applications. Some meeting attendees noted that unreleased versions of CMAQ are already being used for regulatory/SIP modeling.

CMAS has had mixed experience with outside developers and the level of support that they are willing to provide for their codes. Requirement for peer production is that developers must commit to stand behind their codes and provide guidance on testing and using their products.

One issue that has slowed the release of CMAQ through a public repository like source forge is that EPA has not been able to assign a license to the codes.

The issue of attribution in the CMAQ codes was raised with some people complaining that they weren't receiving proper attribution in the codes of some of the CMAQ modules for work that they implemented.

CMAQ needs to have an architect in chief of the codes that is willing to interact with CMAS and the community to decide how to best serve the needs of both the development and user communities. CMAS and the community need more guidance from the EPA development group on how to make CMAQ more of a research/development platform. The interaction between a CMAQ chief architect and the community is integral to keep the software open and be consistent with the community modeling paradigm. The community and CMAS can only do so much without buy in from EPA to help facilitate outside development.

The benefits of the research code archive is that it will give EPA an opportunity to access codes that are deemed good by the community. The research/development version of the model will be a gateway model for the official release version of CMAQ. The research community will still be able to do research with the codes, even if they are not blessed by EPA or CMAS.

There were some attendees who disagreed with the need for a development version of CMAQ. They argued that they actually want to do development on the most recent version of the model and that a development version would lead to a divergence in the model. The point was then raised that for those developing diagnostic tools, like DDM or adjoints, it makes sense to work on the official release version. For more fundamental or architectural changes to the model, like new transport algorithms or variable/adaptive grid routines, there needs to be a more simplified architecture to work on.

There is a need for CMAS to consolidate communication about CMAQ and development. There is some disconnect between the listserv, the bugtracking database, and documentation. There needs to be an effort to consolidate the information in the these resources into a single web site.

Some members of the community have volunteered to make more of an effort to be more proactive through the CMAS wiki.