SMACK: Short Meetings on Astro Computing Knowledge
Talks given in live demostration format on any kind of astronomical software and computing tools for astronomical research.
Short Meetings on Astro Computing Knowledge (SMACK) are a series of talks, or more appropriately 'live demonstrations', presented in the Instituto de Astrofísica de Canarias (IAC), targeting graduate students and researchers. The main aim of the talks is to demonstrate the use and benefits of basic software tools that are commonly required for astronomical research. These talks will be showcasted at IAC Talks and recorded for easy future reference by the community.
The 1st SMACK is a brief introduction to the linux shell and the basic tools that come with it. We'll assume no previous knowledge and include a brief history of the POSIX standard.
The shell (or command-line) is the most commonly used (for advanced operations) interface to Unix-like operating systems.
In this session we'll introduce some of the most commonly used command-line/shell features and how they may be used in a hypothetical research project, using real astronomical data sets.
The session plan (with a listing of used commands and short explanation) is available here: https://gitlab.com/makhlaghi/smack-talks-iac/-/blob/master/smack-2-shell.md
Gnuastro is an official GNU package that is currently maintained at the IAC. It is a large collection of programs to enable easy, robust, fast and efficient data analysis directly on the command-line. For example it can perform arithmetic operations on image pixels or table columns/rows without having to write programs, visualize FITS images as JPG or PDF, convolve an image with a given kernel or matching of kernels, perform cosmological calculations, crop parts of large images (possibly in multiple files), manipulate FITS extensions and keywords, and perform statistical operations. In addition, it contains programs to make catalogs from detection maps, add noise, make mock profiles with a variety of radial functions using monte-carlo integration for their centers, match catalogs, and detect objects in an image among many other operations. Gnuastro is written to comply fully with the GNU coding standards and integrates well with all Unix-like operating systems. This enables astronomers to expect a fully familiar experience in the building, installing and command-line user interaction that they have seen in all the other GNU software that they use (core components in many Unix-like/POSIX operating systems). In this SMACK, we will introduce Gnuastro and demonstrate a selection of its commonly used features. Relevant links are as follows. Lecture-notes: in https://gitlab.com/makhlaghi/smack-talks-iac/-/blob/master/smack-3-gnuastro.md, Gnuastro's main webpage: https://www.gnu.org/s/gnuastro, Gnuastro documentation: https://www.gnu.org/s/gnuastro/manual, Gnuastro tutorials: https://www.gnu.org/s/gnuastro/manual/html_node/Tutorials.html
Zoom link https://rediris.zoom.us/j/94454701469
In the previous SMACKs, basic operations on the command-line were reviewed in interactive mode: where you enter one command, wait for it to finish and enter the next command. The shell's interactive mode is good for small/fast tests, but is not scalable. For example when you know the commands and parameters work and want to apply them to hundreds/thousands of targets. Shell scripts are simply plain-text files that store all the various commands that you want to be executed in sequence. They are designed precisely for the easier management of operations that are more complex than a simple command. In fact many of the commands in the Unix-like operating systems are actually shell scripts! Here will review some basic intricacies with designing robust shell scripts and avoiding common pitfalls. Also, since shell scripts are simple plain-text files, we will also do a short review of simple plain-text editors like GNU Nano and more advanced editors like Vim and GNU Emacs. Advanced editors have many useful features to simplify programming in many languages (including shell programming) and don't need complex graphic environments and can be run in the raw command-line (a major advantage when scaling your project up to supercomputers that don't have a graphic environment). For newcomers to data-intensive astronomical research, we encourage you to select an advanced editor, and master it early to greatly simply your research (in any language).
Git is probably the most widely used Version Control System (software tools that help record changes to files (computer code, text files, etc.) by keeping track of modifications done through time). It can be used for any type of text files, though it is specially useful for programming code and it makes managing your projects, files, and changes made to them much easier and more intuitive.
But it is a big and complex system and people new to it can have a hard time mastering it. In this talk I will introduce the git version control system to people that have never used it before, so I will go over its basic concepts and functionalities from the ground-up. I will cover the most common commands, needed to use Git for your own individual projects. Once you properly understand how to use it on your own, it is much easier to understand how to collaborate with others (for example using GitHub), which will be covered in a follow-up talk: "Intermediate Git".
Following up on our previous "Git version control system basics" seminar (https://bit.ly/35CCX7k), the focus of this talk will be to learn the features of Git that will allow us to collaborate with other colleagues. For this, the main concepts needed are branch merging (including merges with conflicts), remote repositories and hosted repositories (GitHub, GitLab, Bitbucket, etc.).
Thanks to the distributed structure of Git, collaboration with colleagues can become very efficient (allowing for many different workflows) and it avoids the single point of failure of centralized version control systems, but its complexity also goes up. Properly understanding the concept of remote repositories makes collaboration with Git straightforward.
Other Git features and tools that are not essential, but that will make Git usage much more effective and powerful (like stash, rebase, pull requests, etc.) will be left for a planned "Advanced Git" follow-up talk.
This talk will give an overview of the resources for scientific computing at the IAC. The network, los burros, and LaPalma will be described, together with the basics for using them and practical examples.
Speakers: Ana Esteban, Carlos Allende, Isaac Alonso and Juan Carlos Trelles (IAC)
In two previous talks we covered the basics of Git (http://iactalks.iac.es/talks/view/1426) and some intermediate concepts (http://iactalks.iac.es/talks/view/1428). In this one we will focus on more advanced features that can make your Git experience much more productive and efficient.
First we will focus on a very common collaboration "procedure", which we didn't have time to cover in any detail in the "Intermediate Git" talk, namely "Pull requests".
Next, since most of your work with Git will be on local repositories, we will see some more advanced commands and features, such as pull requests, rebase, reset, hunks, reflog, stash and blame.
While this series on Git doesn't cover everything there is to Git, it does cover almost all you'll need to become a regular Git user.
In the last decade, Python went from being a moderated-used programming language in the astronomical community to the de-facto standard in Astronomy. Its recent growth has been spectacular, thanks to the coordination and work of the community to create astropy, the core astronomy library, as well as other base libraries like numpy and matplotlib. However, the current scientific/astronomical ecosystem for Python is huge and sparse, introducing many types of objects and methods, often confusing at the beginning.
In this SMACK talk I will describe the current status of the Python ecosystem for astronomy and introduce the most import elements of the core libraries, numpy and astropy, showing with practical examples how they provide new impressive capabilities to deal with data, catalogs, coordinates and much more, making life easier for astrophysicists.
Pandas is an open source Python package that is widely used for data analysis. It is a powerful ally for data munging/wrangling and databases manipulation/visualisation, and a must-have tool for Data Scientists. In this seminar we will have a general overview on its functionality and we will run over some of the reasons of its large success in the Data Science community.
This talk will present an overview of what Latex is, discuss when to use it and when not, show installing and editing options, introduce classes and packages, and give some usage recommendations and troubleshooting tips.
We will introduce several ways in which trivially embarrassingly parallel tasks can be run in laptops and desktops. We will introduce command-line tools such as GNU parallel and Kiko. We will then focus on simple techniques for optimisation of scientific computations in python. We will cover parallel computing with multiprocessing, acceleration of functions via numba, and GPU computing with cupy. The goal is to provide an easy roadmap for python code optimisation methods that can applied on already existing code, without writing a single line of C or FORTRAN.
Containers are portable environments that package up code and all its dependencies so that an application can run quickly and reliably from one computing environment to another. Most people are probably familiar with full virtualization environments (such as VirtualBox), so in this talk we will explain the main differences between full virtualization and containers (sometimes called light-weight virtualization), and when to use each.
At the same time, not all container technologies have the same goals and/or approaches. Docker is the most mature container offering, but it is geared mainly towards micro-services. Singularity is a newer contender, with an emphasis on mobility of compute for scientific computing. We will introduce both softwares, showing how to create and use containers with each of them, while discussing real-life examples of their use.
The lecture notes can be found here:
- The chemistry of the last major mergerDr. David S. AguadoThursday January 20, 2022 - 10:30 GMT (Aula)
- Seminario de Instrumentación. Progress on IFU development for solar observationsSilvia Regalado OlivaresFriday January 21, 2022 - 12:00 GMT (Aula)