An introduction to computational tools for research-- Linux command line, HPC, Bash scripting, Python, SQL, and Artificial Intelligence
Edit me

BSC4452 and BSC6451 Registration information for Fall 2022

  • BSC 4452: Class 20417

    Image of servers and tool icons

  • BSC 6451: Class 24822

Course Description

The early 2000’s where characterized as the era of Big Data, with researchers across disciplines finding research transformed by large volumes of diverse data. In the past decade, this has been again transformed by the re-emergence of artificial intelligence and machine learning systems aiding in interpretation of Big Data. As data types and volumes continue to grow, knowledge of scripting, database management, and advanced computing skills, including AI fundamentals, are critical for researchers regardless of discipline.

This course introduces students to the tools needed to be proficient, computationally enabled researchers, providing a foundation in Linux and bash scripting for data management, Python coding and basic SQL database fundamentals, and an introduction to artificial intelligence methods.

The course assumes no prior coding or command line skills, and covers concepts that will provide the ability for students to apply new technologies to a wide array of research questions. A foundation in data management and analysis concepts opens doors for well-trained researchers and allows them to work in multidisciplinary fields.

Course Organization

The course is divided into four main sections:

  • Section 1
    • Linux command line
    • Bash scripting
    • Version control using Git and GitHub
    • Using high-performance computing resources
  • Section 2
    • Python scripting
  • Section 3
    • SQL database introduction and integration with Python
  • Section 4 - New for Fall 2022!
    • A brief introduction to Artificial Intelligence

Course Objectives

  • Demonstrate how technology infrastructure can improve research and open new avenues of investigation.
  • Competently navigate the Unix/Linux command line interface.
  • Effectively and efficiently manipulate text files, performing complex regular expression replacements, reformatting and merging files in various ways.
  • Raise and address current issues through class participation and discussion.
  • Use High Performance Computing resources such as the UF Research Computing for cluster-based analyses. Including batch scripting and running multi-processor applications (threaded and MPI).
  • Explain the basic anatomy of computer scripts/programs, with particular focus on Python scripting.
  • Construct analytical pipelines to accomplish complex tasks.
  • Describe basic database design, creation and manipulation. Perform scripted database operations for information discovery, data exploration and research data curation.
  • Have a basic understanding of research graphics formats, preparation and manipulation
  • Have a basic understanding of artificial intelligence and gain hands-on experience with computer vision.

Meeting Times

Synchronous Meetings

  • Mon, Wed, Fri 9:35 AM - 10:25 AM (Period 3)
  • Bartram Hall, room 211
lecture icon
  • I understand that with all that is going on, some students will need to miss classes sometimes. That is fine and I will do my best to help you catch up, but regular attendance is the best way to learn.

Asynchronous Work

  • Students will work on projects either independently or in small, groups.
  • This work can be more flexible in timing and will give you and your classmates times to get more hands-on experience and challenge yourselves to learn on your own.

Student Help Time

Office Hours icon
  • Mondays from 1:00pm to 2:00pm
  • Thursdays from 9:00am to 10:00am
  • By Appointment–please Email me to setup a different time if needed.

Course Textbooks

textbook icon

The main texts for the course are:

Affordable UF badge

Each of these is available as a free PDF download or for purchase in print. Because there are no textbook costs for this course, it has been recognized by Affordable UF as an affordable course.

Course Calendar

calendar icon

For readings, there may be links to pages with my notes and additional explanations on the content from the texts. The texts are abbreviated as TLCL = The Linux Command Line; Py4E = Python for Everyone.

Week Date Reading/Assignment Topic  
1 Wed, Aug 24 Download Software Introduction and course objectives  
1 Fri, Aug 26 Read TLCL Introduction & Ch 1-4
Take UFRC New User Training
Quiz 1 available, due Friday, September 02
Getting started: Computers
UF Research Computing Intro & getting started
 
         
2 Mon, Aug 29 Read TLCL Ch 5-8 Continue building shell skills
Here are some exercises to work on
 
2 Wed, Aug 31 Read Notes on Regular Expressions and TLCL Ch 19
Problem Set 1, due Friday, September 16
Regular Expressions Handout  
2 Fri, Sep 02 Quiz 2 available, due Friday, September 09 Text manipulation  
         
3 Mon, Sep 05   Labor Day, no class  
3 Wed, Sep 07 Read TLCL Ch 20
Optional: Watch Learn the Linux Command Line
GitHub Account assignment due
Read TLCL Ch 24-26
Shell Scripts and version control with git and GitHub  
3 Fri, Sep 09 Quiz 1 due
Quiz 2 due
Continue working on shell scripts and git/github, Github Branching exercise  
         
4 Mon, Sep 12 Read TLCL Ch 27, 29 & 33 Shell Scripts and version control with git and GitHub  
4 Wed, Sep 14 Problem Set 2 available, due Monday, October 03 Git and GitHub.com  
4 Fri, Sep 16 Problem Set 1 due
Watch the HiPerGator: SLURM Submission Scripts training
   
         
5 Mon, Sep 19   Flow control: if, while, until, for etc.  
5 Wed, Sep 21 Read TLCL Ch 23 Compiling source code
Google and Documentation
Using UF Research Computing resources
Running batch jobs
 
5 Fri, Sep 23 Read Py4E Ch 1 Introduction to Python  
         
6 Mon, Sep 26 Problem Set 3 available, due Monday, October 17
Quiz 3 available, due Wednesday, October 05
Read Py4E Ch 2
Python data types  
6 Wed, Sep 28 Read Py4E Ch 3 Python: Flow Control  
6 Fri, Sep 30 Read Py4E Ch 4 Python: Functions  
         
7 Mon, Oct 03 Problem Set 2 due
Read Py4E Ch 5
Python: Iteration  
7 Wed, Oct 05 Quiz 3 due
Read Py4E Ch 6 & Ch 7
Python: try/except, Strings, File I/O  
7 Fri, Oct 07   Homecoming, no class  
         
8 Mon, Oct 10 Read Py4E Ch 8, and Chs 9-10 Lists, Dictionaries, Tuples  
8 Wed, Oct 12 Read Ch 11 RegEx in Python  
8 Fri, Oct 14 Problem Set 4, due Friday, November 04
Read Py4E Ch 12 & Ch 13
Scripting data acquisition  
         
9 Mon, Oct 17 Problem Set 3 due
SciPy, NumPy, Pandas
SciPy, NumPy, Pandas  
9 Wed, Oct 19   Pandas with Messy Data
Data visualization with Pandas
 
9 Fri, Oct 21 Project 1, due Friday, November 04 Matplotlib and data visualization Visualization
         
10 Mon, Oct 24 Quiz 4 available, due Wednesday, November 02 More data visualization  
10 Wed, Oct 26
Scan Py4E Ch 16
Py4E Ch 14: Object oriented Programming
Py4E Ch 14: Object oriented programming  
10 Fri, Oct 28 Problem Set 4 Due No Class, optionally attend UF AI Day  
11 Mon, Oct 31 Programming Foundations Databases Work on Project 1  
11 Wed, Nov 02 Quiz 4 due Database intro
Flight DB Example
 
11 Fri, Nov 04 Project 1 Due
Read Py4E Ch. 15, through 15.5 and my notes
Overview of databases
Database design
 
         
12 Mon, Nov 07 Problem set 5, due Monday, November 21 Py4E Ch. 15, through 15.5 and my notes
Databases, SQL and sqlite
 
12 Wed, Nov 09 Quiz 5 available, due Wednesday, November 30 More on databases and Joins  
12 Fri, Nov 11   Veteran’s Day, no class  
         
13 Mon, Nov 14 Project 2 available, due Wednesday, December 07 SQLAlchemy  
13 Wed, Nov 16   SQLAlchemy and Pandas  
13 Fri, Nov 18   Argparse  
         
14 Mon, Nov 21 Problem Set 5 due Graphics  
14 Wed, Nov 23   Thanksgiving, no Class  
14 Fri, Nov 25   Thanksgiving, no class  
         
15 Mon, Nov 28   Work on Project 2  
15 Wed, Nov 30 Quiz 5 due
Quiz 6 available, due Wednesday, December 07
Work on Project 2  
15 Fri, Dec 02   Intro to AI: ML: Decision trees and XGBoost exercise. Work through this post on Kaggle.  
         
16 Mon, Dec 05   Into to AI  
16 Wed, Dec 07 Project 2 due
Quiz 6 due
Intro to AI  

Software and Hardware

Participants will need a computer with internet connection for all classes.

Several free/open source software packages will be used throughout the course, and students will be required to install some of these.

A (free) Research Computing account will be cxreated for students to access HiPerGator for coursework.

Students will be required to apply for a (free) Github.com account for coursework.

If you have technical difficulties with Canvas, please contact the UF Helpdesk at:

  • http://helpdesk.ufl.edu
  • (352) 392-HELP (4357)
  • Walk-in: HUB 132

Any requests for make-ups due to technical issues should be accompanied by the ticket number received from the Help Desk when the problem was reported to them. The ticket number will document the time and date of the problem. Please e-mail the instructor within 24 hours of the technical difficulty if you wish to request a make-up.

All faculty, staff and student of the University are required and expected to obey the laws and legal agreements governing software use. Failure to do so can lead to monetary damages and/or criminal penalties for the individual violator. Because such violations are also against University policies and rules, disciplinary action will be taken as appropriate.

Grading

See also the List of Graded Work page.

Assignment Values

  • Quizzes: 6 @ 20 points each: 120 points (37%)
  • Problem Sets: 5 @ 20 points each: 100 points (30%)
  • Class Projects: 2 @ 40 points each: 80 points (24%)
  • Class Participation 30 points: 30 points (9%)
    • 5 points for GitHub account assignment
    • 10 points for github commits
    • 15 points for code peer review

Grading in this class is consistent with UF policies available at: https://catalog.ufl.edu/UGRD/academic-regulations/grades-grading-policies/

Should a student wish to dispute any grade received in this class (other than simple addition errors), the dispute must be in writing (via email) and be submitted to the instructor within a week of receiving the grade.

The dispute should clearly set out the grade that the student believes the assignment should have received as well as why they believe that they should have received such a grade.
A A- B+ B B- C+ C C- D+ D D- F
100-93
(4.0)
<93-90
(3.67)
<90-87
(3.33)
<87-83
(3.0)
<83-80
(2.67)
<80-77
(2.33)
<77-73
(2.0)
<73-70
(1.67)
<70-67
(1.33)
<67-63
(1.0)
<63-60
(0.67)
<60
(0)
Note: A grade of C- is not a qualifying grade for major, minor, Gen Ed, or College Basic distribution credit. For further information on UF's Grading Policy, see: https://catalog.ufl.edu/UGRD/academic-regulations/grades-grading-policies/

Course Policies

Class Attendance and Makeup Policy

Requirements for class attendance and makeup assignments, and other work in this course are consistent with university policies that can be found in the online catalog at: https://catalog.ufl.edu/UGRD/academic-regulations/attendance-policies/

In general, I do not take attendance. You are all adults and I assume you are taking the class the learn. The best way to learn is to regularly attend class. I am sure students will miss class for various reasons. I am happy to help you catch up. If you regularly miss class and fall behind, I may ask that you hold questions on content you have missed until after class, or ask that you coordinate a time to go over the content. I will make every effort to record and post all classes to help those that miss classes.

Quiz and Assignment Policy

Quiz and assignment dates will be announced at least one week in advance and students will have at least three days to complete the quiz or assignment. Each quiz or assignment will clearly state if it is an individual or group assignment. Individual assignments must be the student’s own work, completed without the assistance of others.

All quizzes and assignments are “open book, open internet”, you may use whatever resources you desire to complete the quiz/assignment. Though only assignments specifically noted as group assignments should be worked on with other people.

Makeup and Late policy

Please notify the instructor of circumstances that lead to late work or missed classes. I will generally work with you and accept late work. Without prior notification, late work will be penalized one point per day after the due date.

In response to COVID-19, the following recommendations are in place to maintain your learning environment, to enhance the safety of our in-classroom interactions, and to further the health and safety of ourselves, our neighbors, and our loved ones.

  • If you are not vaccinated, get vaccinated. Vaccines are readily available and have been demonstrated to be safe and effective against the COVID-19 virus. Visit one.uf for screening / testing and vaccination opportunities.
  • If you are sick, stay home. Please call your primary care provider if you are ill and need immediate care or the UF Student Health Care Center at 352-392-1161 to be evaluated.
  • Course materials will be provided to you with an excused absence, and you will be given a reasonable amount of time to make up work.

Students Requiring Accommodations

Students with disabilities requesting accommodations should first register with the Disability Resource Center (352-392-8565, https://disability.ufl.edu/students/get-started/) by providing appropriate documentation. Once registered, students will receive an accommodation letter which must be presented to the instructor when requesting accommodation. Students with disabilities should follow this procedure as early as possible in the semester.

Course Evaluation

Students are expected to provide professional and respectful feedback on the quality of instruction in this course by completing course evaluations online via GatorEvals. Guidance on how to give feedback in a professional and respectful manner is available at gatorevals.aa.ufl.edu/students/. Students will be notified when the evaluation period opens, and can complete evaluations through the email they receive from GatorEvals, in their Canvas course menu under GatorEvals, or via ufl.bluera.com/ufl/. Summaries of course evaluation results are available to students at gatorevals.aa.ufl.edu/public-results/.

Class Demeanor and Netiquette

Students are expected behave in a manner that is respectful to the instructor and to fellow students. Opinions held by other students should be respected in discussion, and conversations that do not contribute to the discussion should be held at minimum, if at all.

Students should be working on course content during class.

GitHub Discussion Board

Canvas can be a challenge when working with code. As such we will use the GitHub discussion board to ask for and provide help by all. Students should be supportive and considerate of others at all times. Rude or inappropriate comments will be removed and the poster will be warned.

University Honesty Policy

UF students are bound by The Honor Pledge which states:

We, the members of the University of Florida community, pledge to hold ourselves and our peers to the highest standards of honor and integrity by abiding by the Honor Code. On all work submitted for credit by students at the University of Florida, the following pledge is either required or implied: “On my honor, I have neither given nor received unauthorized aid in doing this assignment.”

The Honor Code specifies a number of behaviors that are in violation of this code and the possible sanctions. Furthermore, you are obligated to report any condition that facilitates academic misconduct to appropriate personnel. If you have any questions or concerns, please consult with the instructor or TAs in this class

Resources are available on-campus for students having personal problems or lacking clear career and academic goals. The resources include:
  • UF Counseling & Wellness Center, 3190 Radio Rd, 392-1575, psychological and psychiatric services.
    • Provides counseling and support as well as crisis and wellness services including a variety of workshops throughout the semester (e.g., Yappy Hour, Relaxation and Resilience).
    • Many students experience test anxiety and other stress related problems. “A Self Help Guide for Students” is available through the Counseling Center (301 Peabody Hall, 392-1575) and at their web site: https://counseling.ufl.edu/.
    • U Matter, We Care: If you or a friend is in distress, please contact umatter@ufl.edu or 352 392-1575 so that a team member can reach out to the student.
  • Career Connections Center, Reitz Union, 392-1601, CareerCenterMarketing@ufsa.ufl.edu, connects job seekers with employers and offers guidance to enrich your collegiate experience and prepare you for life after graduation.
  • University Police Department: 392-1111 or 9-1-1 for emergencies. https://www.police.ufl.edu/
  • Sexual Assault Recovery Services (SARS): Student Health Care Center, 392-1161.
  • Student Health Care Center: Call 352-392-1161 for 24/7 information to help you find the care you need, or visit https://shcc.ufl.edu/
  • Food insecurity: The Pantry is a resource on the University of Florida campus committed to supporting students, staff, and faculty who are experiencing food insecurity. These individuals do not have reliable access to nutritious foods for themselves and their families. If you, or anyone you know, is experiencing food insecurity, the Pantry is a resource to visit. We offer non-perishable food, toiletries and fresh produce grown at the Field and Fork Gardens during certain times of the year. There is no proof of need required in order to shop at the Pantry; you must only bring in your valid UFID card. At the Pantry, we know that a good meal makes for a good student, and we work to support all Gators who are experiencing food insecurity. Field & Fork Food Pantry.

Inclusive Learning Environment

This course embraces the University of Florida’s Non-Discrimination Policy, which reads:

The University shall actively promote equal opportunity policies and practices conforming to laws against discrimination. The University is committed to nondiscrimination with respect to race, creed, color, religion, age, disability, sex, sexual orientation, gender identity and expression, marital status, national origin, political opinions or affiliations, genetic information and veteran status as protected under the Vietnam Era Veterans’ Readjustment Assistance Act.

If you have questions or concerns about your rights and responsibilities for inclusive learning environment, please see the instructor or refer to the Office of Multicultural & Diversity Affairs website: http://multicultural.ufl.edu.

Privacy

There are federal laws protecting your privacy with regards to grades earned in courses and on individual assignments. For more information, please see: https://registrar.ufl.edu/ferpa.html

Statement Regarding Course Recording

Our class sessions may be audio visually recorded for students in the class to refer back to and for use of enrolled students who are unable to attend live. Students who participate with their camera engaged or utilize a profile image are agreeing to have their video or image recorded. If you are unwilling to consent to have your profile or video image recorded, keep your camera off and do not use a profile image. Likewise, students who un-mute during class and participate verbally are agreeing to have their voices recorded.  If you are unwilling to consent to have your voice recorded during class, you will need to keep your mute button activated and communicate exclusively using the “chat” feature, which allows students to type questions and comments live. The chat will not be recorded or shared. As in all courses, unauthorized recording and unauthorized sharing of recorded materials is prohibited.

Disclaimer on Free and Open Discussion

Students are encouraged to employ critical thinking and to rely on data and verifiable sources to interrogate all assigned readings and subject matter in this course as a way of determining whether they agree with their classmates and/or their instructor. No lesson is intended to espouse, promote, advance, inculcate, or compel a particular feeling, perception, viewpoint or belief.

Dean of Students Office

Dean of Students Office (352-392-1261) provides a variety of services to students and families, including Field and Fork (UF’s food pantry) and New Student and Family programs</p>

Disability Resource Center

  • Disability Resource Center (DRCaccessUF@ufsa.ufl.edu | 352-392-8565) helps to provide an accessible learning environment for all by providing support services and facilitating accommodations, which may vary from course to course. Once registered with DRC, students will receive an accommodation letter that must be presented to the instructor when requesting accommodations. Students should follow this procedure as early as possible in the semester.

Multicultural and Diversity Affairs

Multicultural and Diversity Affairs (352-294-7850) celebrates and empowers diverse communities and advocates for an inclusive campus.

Office of Student Veteran Services

Office of Student Veteran Services (352-294-2948 | vacounselor@ufl.edu) assists student military veterans with access to benefits.

ONE.UF

ONE.UF is the home of all the student self-service applications, including access to:

Official Sources of Rules and Regulations

The official source of rules and regulations for UF students is the Undergraduate Catalog and Graduate Catalog. Quick links to other information have also been provided below.