Publications
300x faster Matlab using MatlabMPI
July 18, 2002
Journal Article
Published in:
https://arxiv.org/abs/astro-ph/0207389
Topic:
R&D group:
Summary
The true costs of high performance computing are currently dominated by software. Addressing these costs requires shifting to high productivity languages such as Matlab. MatlabMPI is a Matlab implementation of the Message Passing Interface (MPI) standard and allows any Matlab program to exploit multiple processors. MatlabMPI currently implements the basic six functions that are the core of the MPI point-to-point communications standard. The key technical innovation of MatlabMPI is that it implements the widely used MPI "look and feel" on top of standard Matlab file I/O, resulting in an extremely compact (~250 lines of code) and "pure" implementation which runs anywhere Matlab runs, and on any heterogeneous combination of computers. The performance has been tested on both shared and distributedmemory parallel computers (e.g. Sun, SGI, HP, IBM and Linux). MatlabMPI can match the bandwidth of C based MPI at large message sizes. A test image filtering application using MatlabMPI achieved a speedup of ~300 using 304 CPUs and ~15% of the theoretical peak (450 Gigaflops) on an IBM SP2 at the Maui High Performance Computing Center. In addition, this entire parallel benchmark application was implemented in 70 software-lines-of-code (SLOC) yielding 0.85 Gigaflop/SLOC or 4.4 CPUs/SLOC, which are the highest values of these software price performance metrics ever achieved for any application. The MatlabMPI software will be made available for download.
Summary
The true costs of high performance computing are currently dominated by software. Addressing these costs requires shifting to high productivity languages such as Matlab. MatlabMPI is a Matlab implementation of the Message Passing Interface (MPI) standard and allows any Matlab program to exploit multiple processors. MatlabMPI currently implements the basic...
READ MORE
Detecting clusters of galaxies in the Sloan Digital Sky Survey. I. Monte Carlo comparison of cluster detection algorithms
January 1, 2002
Journal Article
Published in:
Astron. J., Vol. 123, No. 1, January 2002, pp. 20-36.
Topic:
R&D group:
Summary
We present a comparison of three cluster-finding algorithms from imaging data using Monte Carlo simulations of clusters embedded in a 25 deg(2) region of Sloan Digital Sky Survey (SDSS) imaging data: the matched filter (MF), the adaptive matched filter (AMF), and a color-magnitude filtered Voronoi tessellation technique (VTT). Among the two matched filters, we find that the MF is more efficient in detecting faint clusters, whereas the AMF evaluates the redshifts and richnesses more accurately, therefore suggesting a hybrid method (HMF) that combines the two. The HMF outperforms the VTT when using a background that is uniform, but it is more sensitive to the presence of a nonuniform galaxy background than is the VTT; this is due to the assumption of a uniform background in the HMF model. We thus find that for the detection thresholds we determine to be appropriate for the SDSS data, the performance of both algorithms are similar; we present the selection function for each method evaluated with these thresholds as a function of redshift and richness. For simulated clusters generated with a Schechter luminosity function (M(*r) = -21.5 and (a = -1.1), both algorithms are complete for Abell richness >~ clusters up to z ~0.4 for a sample magnitude limited to r = 21. While the cluster parameter evaluation shows a mild correlation with the local background density, the detection efficiency is not significantly affected by the background fluctuations, unlike previous shallower surveys.
Summary
We present a comparison of three cluster-finding algorithms from imaging data using Monte Carlo simulations of clusters embedded in a 25 deg(2) region of Sloan Digital Sky Survey (SDSS) imaging data: the matched filter (MF), the adaptive matched filter (AMF), and a color-magnitude filtered Voronoi tessellation technique (VTT). Among the...
READ MORE
PVL: An Object Oriented Software Library for Parallel Signal Processing (Abstract)
January 1, 2002
Conference Paper
Published in:
CLUSTER '01, 2001 IEEE Int. Conf. on Cluster Computing, 8-11 October 2001, p. 74.
Topic:
R&D group:
Summary
Real-time signal processing consumes the majority of the world's computing power Increasingly, programmable parallel microprocessors are used to address a wide variety of signal processing applications (e.g. scientific, video, wireless, medical, communication, encoding, radar, sonar and imaging). In programmable systems the major challenge is no longer hardware but software. Specifically, the key technical hurdle lies in mapping (i.e., placement and routing) of an algorithm onto a parallel computer in a general manner that preserves software portability. We have developed the Parallel Vector Library (PVL) to allow signal processing algorithms to be written using high level Matlab like constructs that are independent of the underlying parallel mapping. Programs written using PVL can be ported to a wide range of parallel computers without sacrificing performance. Furthermore, the mapping concepts in PVL provide the infrastructure for enabling new capabilities such as fault tolerance, dynamic scheduling and self-optimization. This presentation discusses PVL with particular emphasis on quantitative comparisons with standard parallel signal programming practices.
Summary
Real-time signal processing consumes the majority of the world's computing power Increasingly, programmable parallel microprocessors are used to address a wide variety of signal processing applications (e.g. scientific, video, wireless, medical, communication, encoding, radar, sonar and imaging). In programmable systems the major challenge is no longer hardware but software. Specifically...
READ MORE
Parallel programming with MatlabMPI
July 20, 2001
Journal Article
Published in:
https://arxiv.org/abs/astro-ph/0107406
Topic:
R&D group:
Summary
MatlabMPI is a Matlab implementation of the Message Passing Interface (MPI) standard and allows any Matlab program to exploit multiple processors. MatlabMPI currently implements the basic six functions that are the core of the MPI point-to-point communications standard. The key technical innovation of MatlabMPI is that it implements the widely used MPI "look and feel" on top of standard Matlab file I/O, resulting in an extremely compact (~100 lines) and "pure" implementation which runs anywhere Matlab runs. The performance has been tested on both shared and distributed memory parallel computers. MatlabMPI can match the bandwidth of C based MPI at large message sizes. A test image filtering application using MatlabMPI achieved a speedup of ~70 on a parallel computer.
Summary
MatlabMPI is a Matlab implementation of the Message Passing Interface (MPI) standard and allows any Matlab program to exploit multiple processors. MatlabMPI currently implements the basic six functions that are the core of the MPI point-to-point communications standard. The key technical innovation of MatlabMPI is that it implements the widely...
READ MORE
High Speed Interconnects and Parallel Software Libraries: Enabling Technologies for NVO
January 1, 2001
Conference Paper
Published in:
Proc. of the Astronomical Society of the Pacific Conf. Series, Vol. 225, 2001, Virtual Observations of the Future, 13-16 June 2000, pp. 297-301.
R&D group:
Summary
The National Virtual Observatory (NVO) will directly or indirectly touch upon all steps in the process of transforming raw observational data into "meaningful" results. These steps include: (1) Acquisition and storage of raw data. (2) Data reduction (i.e. translating raw data into source detections). (3) Aquisition and storage of detected sources. (4) Multi-sensor/multi-temporal data mining of the products of steps (1), (2) and (3). (Not complete.)
Summary
The National Virtual Observatory (NVO) will directly or indirectly touch upon all steps in the process of transforming raw observational data into "meaningful" results. These steps include: (1) Acquisition and storage of raw data. (2) Data reduction (i.e. translating raw data into source detections). (3) Aquisition and storage of detected...
READ MORE
Exploiting VSIPL and OpenMP for Parallel Image Processing
January 1, 2001
Conference Paper
Published in:
ADASS 2000, Astronomical Data Analysis Software and Systems X, 12-14 November 2000, pp. 209-212.
R&D group:
Summary
VSIPL and OpenMP are two open standards for portable high performance computing. VSIPL delivers optimized single processor performance while OpenMP provides a low overhead mechanism for executing thread based parallelism on shared memory systems. Image processing is one of the main areas where VSIPL and OpenMP can have a large impact. Currently, a large fraction of image processing applications are written in the Interpreted Data Language (IDL) environment. The aim of this work is to demonstrate that the performance benefits of these new standards can be brought to image processing community in a high level manner that is transparent to users. To this end, this talk presents a fast, FFT based algorithm for performing image convolutions. This algorithm has been implemented within the IDL environment using VSIPL (for optimized single processor performance) with added OpenMP directives (for parallelism). This work demonstrates that good parallel speedups are attainable using standards and can be integrated seamlessly into existing user environments.
Summary
VSIPL and OpenMP are two open standards for portable high performance computing. VSIPL delivers optimized single processor performance while OpenMP provides a low overhead mechanism for executing thread based parallelism on shared memory systems. Image processing is one of the main areas where VSIPL and OpenMP can have a large...
READ MORE
Cluster Computing for Embedded/Real-Time Systems
January 1, 2000
Book Chapter
Published in:
Cluster Computing White Paper
Topic:
R&D area:
R&D group:
Summary
Cluster computing is not a new area of computing. It is, however, evident that there is agrowing interest in its usage in all areas where applications have traditionally used parallelor distributed computing platforms. The mounting interest has been fuelled in part by theavailability of powerful microprocessors and high-speed networks as off-the-shelf commoditycomponents as well as in part by the rapidly maturing software components available tosupport high performance and high availability applications.This rising interest in clusters led to the formation of an IEEE Computer Society Task Forceon Cluster Computing (TFCC1) in early 1999. An objective of the TFCC was to act both as amagnet and a focal point for all cluster computing related activities. As such, an earlyactivity that was deemed necessary was to produce a White Paper on cluster computing andits related technologies.Generally a White Paper is looked upon as a statement of policy on a particular subject. Theaim of this White Paper is to provide a relatively unbiased report on the existing, new andemerging technologies as well as the surrounding infrastructure deemed important to thecluster computing community. This White Paper is essentially a snapshot of cluster-relatedtechnologies and applications in year 2000.This White Paper provides an authoritative review of all the hardware and softwaretechnologies that can be used to make up a cluster now or in the near future. Thesetechnologies range from the network level, through the operating system and middlewarelevels up to the application and tools level. The White Paper also tackles the increasinglyimportant areas of High Availability and Embedded/Real Time applications, which are bothconsidered crucial areas for future clusters.The White Paper has been broken down into twelve chapters, each of which has been puttogether by academics and industrial researchers who are both experts in their fields andwhere willing to volunteer their time and effort to put together this White Paper.On a personal note, I would like to thank all the contributing authors for finding the time toput the effort into their chapters and making the overall paper an excellent state-of-the-artreview of clusters. In addition, I would like to thank the reviewers for their timely comments.
Summary
Cluster computing is not a new area of computing. It is, however, evident that there is agrowing interest in its usage in all areas where applications have traditionally used parallelor distributed computing platforms. The mounting interest has been fuelled in part by theavailability of powerful microprocessors and high-speed networks as...
READ MORE
The Vector, Signal, and Image Processing Library (VSIPL): an Open Standard for Astronomical Data Processing
December 1, 1999
Abstract
Published in:
Bulletin of the American Astronomical Society, Vol. 31, p.1497
Topic:
R&D area:
R&D group:
Summary
The Vector/Signal/Image Processing Library (VSIPL) is a DARPA initiated effort made up of industry, government and academic representatives who have defined an industry standard API for vector, signal, and image processing primitives for real-time signal processing on high performance systems. VSIPL supports a wide range of data types (int, float, complex, ...) and layouts (vectors, matrices and tensors) and is ideal for astronomical data processing. The VSIPL API is intended to serve as an open, vendor-neutral, industry standard interface. The object-based VSIPL API abstracts the memory architecture of the underlying machine by using the concept of memory blocks and views. Early experiments with VSIPL code conversions have been carried out by the High Performance Computing Program team at the UCSD. Commercially, several major vendors of signal processors are actively developing implementations. VSIPL has also been explicitly required as part of a recent Rome Labs teraflop procurement. This poster presents the VSIPL API, its functionality and the status of various implementations.
Summary
The Vector/Signal/Image Processing Library (VSIPL) is a DARPA initiated effort made up of industry, government and academic representatives who have defined an industry standard API for vector, signal, and image processing primitives for real-time signal processing on high performance systems. VSIPL supports a wide range of data types (int, float...
READ MORE