High Performance Computing


Feb 01, 2008 Print Article

High-performance computing (HPC) integrates systems administration (including network and security knowledge) and parallel programming into a multidisciplinary field that combines digital electronics, computer architecture, system software, programming languages, algorithms and computational techniques. Formerly the domain of scientific research, the technology has shifted from supercomputers to clusters and grids of commercial off-the-shelf microcomputers, and thus moving HPC into the mainstream marketplace through business, education, government and the military. This shift has, in turn, created a demand for HPC technicians, who are in short supply. Colleges should incorporate HPC principles into existing and related IT programs and consider developing new curriculum to meet this growing industry demand.

Performance Indicators

  • Timing: HPC is deployed today in many research institutions and datacenter environments. HPC technicians are in short supply; however, the employer base is hard to identify.1 Colleges with robust computer information and/or computer science programs that are co-located near organizations with HPC environments should investigate the feasibility of an advanced certificate in HPC.
  • Relevance: HPC skills are an extension of traditional skills taught in many colleges. Faculty qualifications are important to the successful implementation of an HPC program.2 HPC can also be a costly program to implement. Cost burden may be offset by focusing on HPC programming and by using On-Demand HPC. For example, Nano-Hub enables anyone to download a free HPC grid appliance. The appliance enables one to run a development “virtual workspace” on a desktop and to deploy custom applications to the Nano-Hub Web site at www.nanohub.org.
  • Jobs: The pay range for HPC related occupations is $30,000 to $120,000 per year.3 Entry-level positions available to college graduates include application developers, test engineers, systems administrators and field support engineers.
  • Trends: During the past decade, HPC has migrated from almost exclusively scientific research to diversified industries, including commercial electronics, energy (petroleum, oil and gas), chemical, pharmaceutical, biomedical, life science, aerospace, automotive, entertainment, telecommunications, transportation, financial and defense.4
  • Timing: HPC is deployed today in many research institutions and enterprise datacenter environments. HPC technicians are in short supply;5 however, the employer base is difficult to identify.6 Colleges with robust computer information and/or computer science programs that are co-located near organizations with HPC environments should update existing curriculum to include HPC principles and consider developing advanced certificates specializing in HPC. Colleges should monitor HPC, as this field of computing has the potential to grow into a large industry that redefines enterprise class computing between 2007-2012.
  • Relevance: HPC is relevant to colleges that have a robust IT and/or computer science curriculum and a local employment base requiring HPC technicians. Students graduating with HPC skills will have job opportunities across education, military, government and industry sectors.

Defining High-Performance Computing

High-performance computing (HPC) is not new but is moving from the scientific laboratory to the mainstream. A few examples of commercial HPC are the simulation of car crashes for structural design, molecular interaction for new drug design and the airflow over automobiles or airplanes. In government and research institutions, scientists are simulating galaxy creation, fusion energy and global warming, as well as working to create more accurate short- and long-term weather forecasts. Entertainment companies may use HPC to render graphics and animations for new movies and video games.7

HPC is a multidisciplinary field, combining digital electronics, computer architecture, system software, programming languages, algorithms and computational techniques8 in parallel computing environments to model complex systems9 or to control transactional computing services.10 HPC is a term applied to many different technologies11 and it is rapidly evolving.12 In the past, HPC was most frequently associated with scientific research; however, HPC has been applied recently to business, government and military uses of cluster-based computing strategies, such as data warehouses, line-of-business (LOB) applications, transaction processing, utility computing and next generation telecommunications.13

The two primary areas of specialization for HPC available to college graduates are systems administration (requiring network and security knowledge) and parallel programming.

Employability

The pay range for HPC is $30,000 to $120,000 per year.14 Positions available to college graduates include application developers, test engineers and field support engineers. Based on survey findings and employment projections, there will be a minimum of 164,397 HPC positions in the U.S. for which associate degree holders will be eligible between 2001 and 2008.15 HPC technicians may find employment at national laboratories, universities, businesses, government and military organizations.

Skill Sets

The two primary areas of specialization for HPC available to college graduates are systems administration (requiring network and security knowledge) and parallel programming.HPC introduces new techniques related to “concurrency” and “latency” that increase the complexity of systems administration, security, network and programming.16 HPC employment opportunities17 for college graduates include:

  • Application Developers: Developers should know the principles of parallel programming, parallel tools, and be able to build thread-safe component interfaces.
  • Systems Administrators: Administrators should have experience with HPC system management tools, HPC file systems and HPC batch schedulers. Administrators should also be familiar with security issues in HPC environments.
  • Test Engineers: Test engineers should have parallel debugging skills and they should be familiar with parallel analysis and profiling tools.
  • Field Support Engineers: Field Support engineers need some parallel debugging skills and should also have knowledge of parallel tools.

A comprehensive program preparing an HPC technician should include elements of conventional Linux, network, and security administration, as well as specialized training focused on cluster specific administration skills.18 The foundation for HPC skills is found in college curricula related to the following certifications: Cisco Certified Internetwork Expert (CCIE), Security+, Red Hat Certified Engineer (RHCE), Cisco Certified Network Professional (CCNP), Certified Information Systems Security Professional (CISSP), Check Point Certified Security Administrator (CCSA), Linux+, Microsoft Certified Systems Administrator (MCSA), Sun Certified System Administrator for Solaris Operating Environment, Citrix Certified Enterprise Administrator (CCEA), and Microsoft Certified Database Administrator (MCDBA).19

Trends

During the past 10 years, HPC has shifted from proprietary supercomputers to open systems. As a result, the technology has migrated from almost exclusively scientific research to diversified industries, including commercial electronics, energy (petroleum, oil, and gas), chemical, pharmaceutical, biomedical, life science, aerospace, automotive, entertainment, telecommunications, transportation, financial and defense.20

Drivers

Over the past decade, HPC has migrated from supercomputers to “clusters” and “grids” of Commercial-Off-The-Shelf (COTS) microcomputers (uni-core and multi-core). Seventy-five percent of the top 500 HPC installations in June 2007 were clusters, compared to 21 percent massively parallel (monolithic supercomputers) and 4 percent constellations (grid).21 The driver for this move is a combination of Moore’s Law (enabling higher performance computers at lower costs) and the acceptance of open systems software by the HPC community. Linux is the OS of preference for HPC systems.22

Once the domain of scientists and researchers, HPC is now moving into the mainstream to replace multi-million dollar mainframes and supercomputers with networks and clusters of microcomputers acting in unison to deliver high end computing services. As HPC moves deeper into the enterprise marketplace, the applications served by these machines have bifurcated into classes of systems important to employers and colleges: (1) modeling, analysis and prediction, and (2) enterprise-class transactional computing. Three drivers of modeling, analysis and prediction are the adoption of HPC as a competitive tool by American industry,23 increased use of HPC by the military and intelligence communities24 and new offerings related to “On-Demand HPC,” which delivers HPC as a service on a “time-share” basis. Three drivers of enterprise-class transactional computing include relative cost per Tera Flop, enhanced scalability and support for open systems (as opposed to proprietary systems).

A comprehensive program preparing an HPC technician should include elements of conventional Linux, network, and security administration, as well as specialized training focused on cluster specific administration skills.

HPC deployment is complex and expensive. Many HPC implementations are associated with “green field” business opportunities, rather than “legacy” business opportunities. As a result, HPC growth will be a function of the marketplace’s ability to monetize applications associated with HPC.25

Availability of human expertise (and the budget to hire expertise) associated with HPC is the largest single factor holding HPC back in many corporate environments. Closely related is the ease-of-use issue; most industrial sites require software compatibility in their HPC servers, and the cost to change or rewrite software is frequently seen as prohibitive.26

Sources

  1. McBride, op. cit.; Murphy, op. cit.
  2. Murphy, op. cit.
  3. Murphy, op. cit.
  4. Joseph, op. cit.
  5. McBride, op. cit.
  6. Murphy, op. cit.
  7. Schulman, Michael A. “High-Performance Computing: RAM Versus CPU.” Dr. Dobb’s Portal. 30 April 2007. Retrieved 30 August 2007 www.ddj.com/hpc-high-performance-computing/199201209
  8. “High Performance Computing, Concept, Method, and Means.” Department of Computer Science and Center for Computation and Technology at Louisiana State University. Retrieved 30 August 2007 www.cct.lsu.edu/csc7600/
  9. Joseph, Earl, PhD., Addison Snell, and Christopher G. Willard, PhD. “White Paper: Council on Competitiveness Study of U.S. Industrial HPC Users.” Council on Competitiveness. July 2004. Retrieved 30 August 2007 www.compete.org/pdf/HPC_Users_Survey.pdf
  10. Fussell, Don. Phone interview. Trammell Crow Professor of Computer Science, University of Texas at Austin. 30 August 2007.
  11. McBride, Duncan. Phone interview. Program Director, Division of Undergraduate Education, National Science Foundation. 28 August 2007.
  12. Strohmaier, Erich and Hans W. Meuer. “Supercomputing: What have we learned from the TOP500 Project?” Top 500 Supercomputer Sites. Lawrence Berkeley National, Laboratory, Future Technologies Group. Retrieved 30 August 2007 http://ftg.lbl.gov/ToP500/SP Comput-learned.pdf
  13.  Ward, Dan. Phone interview. West/Midwest SystemsEngineering Manager, Sun Microsystems. 28 August 2007.
  14. Murphy, Tom. Phone interview. Chair, Department of Computer Science, Contra Costa College. 28 August 2007.
  15. “Application for Approval – New Occupational Program: High Performance Computing.” Contra Costa College California Community Colleges. 2003. Retrieved 30 August 2007 www.contracosta.edu/hpc/resources/curriculum/state_program/HPCProgram.doc
  16. Fussell, op. cit.
  17. Apiki, Steve. “Managing Multi-Core Projects, Part 1: HPC on the Parallelism Frontier.” DevX. 5 December 2006. Retrieved 30 August 2007 www.devx.com/go-parallel/Article/33121
  18. Murphy, op. cit.
  19.  Application, op. cit.
  20.  Joseph, op. cit.
  21. “Architecture share for 06/2007.” Top 500 Supercomputer Sites. Retrieved 30 August 2007 www.top500.org/charts/list/29/archtype
  22.  “Operating System share for 06/2007.” Top500 Supercomputer Sites. Retrieved 30 August 2007 www.top500.org/charts/list/29/os
  23. Joseph, op. cit.
  24. Ward, op. cit.
  25.  Ward, op. cit.
  26.  Joseph, op. cit.
TechBriefs
Comments are closed.