Syllabus
Course materials
- Readings are provided via hyperlinks under Schedule.
- Exercises, code, etc. will be posted on Canvas.
- For your Python distribution, I suggest the Anaconda distribution of Python 3.
Course overview and objectives
Human language technology, to which large language models like ChatGPT belong, has made amazing strides. Such technology lets us do things like automatically translate from one language to another, analyze what people on social media think about a current topic, or even write code with a copilot. Machine learning plays a central role in this technology: software that can learn from experience. This course provides an overview of basic methods for machine learning and natural language processing. This is a very hands-on course in which we are going to be using the Python programming language.
The first half of this course focuses on basic machine learning algorithms and applying them to analyze data, much of which is textual. The later portion of the course will shift to natural language processing and to class projects, which will be a major component of the course grade. These projects will allow you to pursue your own interests (and conduct new research in so doing!).
Topics of this course include:
- Python programming for data analysis (including relevant libraries for modeling + data munging)
- Text processing and workflows for text analysis
- Supervised and unsupervised learning
- Natural language processing and applications
Acknowledgements: This course was developed by Jessy Li.
- Four homeworks, 60%
- These will involve programming, and you will be required to turn in your code.
- Class project, 40%
- This will be an open-ended project about a predictive model you would like to build, or a task you would like to accomplish (e.g., predicting the emotion of a Reddit post, summarizing news documents, etc.). Work with the instructor to find a topic that (importantly!) you are interested in or excited about.
- Course projects should be done by teams of 2 students. Project groups consisting of 1 or 3 students are possible only with prior approval of the instructor.
- Milestones:
- An initial presentation and a 2-page written proposal of your project;
- Final project presentation;
- Project writeup (4 pages) and source code.
- The course will use plus-minus grading, using the following scale:
| Grade |
Percentage |
| A |
>= 93% |
| A- |
>= 90% |
| B+ |
>= 87% |
| B |
>= 83% |
| B- |
>= 80% |
| C+ |
>= 77% |
| C |
>= 73% |
| C- |
>= 70% |
| D+ |
>= 67% |
| D |
>= 63% |
| D- |
>= 60% |
Extension policy
- Slip days: You will have a total of 6 free slip days that you can use throughout the semester. You can choose how many days you want to use, and however you want to distribute them; e.g., you can use one slip day per assignment, or 4 days for one assignment and none for others. Slip days cannot be used fractionally: submitting an assignment 1 hour late incurs 1 slip day, 25 hours late incurs 2 slip days, etc.
- Extension permissions: Beyond the slip days, extensions may be granted on a case-by-case basis due to a medical emergency or other circumstances that are extraordinary or emergencies in nature. You must reach out to the instructor to obtain an extension before the deadline in question.
- Late penalty: If you have used up your free slip days and did not obtain a permission on extension, then by default, 10 points (out of 100) will be deducted for lateness, plus an additional 5 points for every 24-hour period beyond 2 that the assignment is late. Resubmissions of assignments are allowed; extension penalty applies for post-deadline resubmissions.
Artificial Intelligence policy
The use of generative artificial intelligence tools (or Large Language Models [LLMs]) such as CoPilot or ChatGPT in this class is strictly prohibited. This includes using AI to generate ideas, outline an approach, answer questions, solve problems, or create original language. All work in this course must be your own or created in group work, where allowed.
Academic dishonesty policy
You are encouraged to discuss assignments with classmates. But all coding/written work must be your own. Students caught cheating will automatically fail the course. If in doubt, ask the instructor.
Notice about students with disabilities
The university is committed to creating an accessible and inclusive learning environment for students with disabilities consistent with university policy and federal and state law. Please let me know if you experience any barriers to learning so I can work with you to ensure you have equal opportunity to participate fully in this course. If you are a student with a disability, or think you may have a disability, and need accommodations please contact Disability and Access (D&A). Please refer to D&A’s website for contact and more information: http://disability.utexas.edu/. If you are already registered with D&A , please deliver your accommodation letter to me as early as possible in the semester so we can discuss your approved accommodations and needs in this course.
Notice about missed work due to religious holy days
By UT Austin policy, you must notify the instructor of your pending absence for a religious holy day as far in advance as possible of the date of observance. If you must miss a class, an examination, a work assignment, or a project in order to observe a religious holy day, you will be given an opportunity to complete the missed work within a reasonable time after the absence. For questions regarding religious accommodations, please contact the Office of the Dean of Students.
Senate Bill 212 and Title IX Reporting Requirements
Under Senate Bill 212 (SB 212), the instructor and TA for this course are required to report for further investigation any information concerning incidents of sexual harassment, sexual assault, dating violence, and stalking committed by or against a UT student or employee. Federal law and university policy also require reporting incidents of sex- and gender-based discrimination and sexual misconduct (collectively known as Title IX incidents). This means we cannot keep confidential information about any such incidents that you share with us. If you need to talk with someone who can maintain confidentiality, please contact University Health Services (512-471-4955 or 512-475-6877) or the UT Counseling and Mental Health Center (512-471-3515 or 512-471-2255). We strongly urge you make use of these services for any needed support and that you report any Title IX incidents to the Title IX Office.
Sharing of Course Materials is Prohibited
No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class without my explicit, written permission. Unauthorized sharing of materials may facilitate cheating. The University is aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in initiation of the student conduct process and include charge(s) for academic misconduct, potentially resulting in sanctions, including a grade impact.
Schedule
Schedule is tentative and subject to change.
- Week 1 (1/12-1/16)
- Introduction (slides)
- Python review
- Probability review
- 1/16: Homework 0 due midnight (hard deadline)
- Week 2 (1/21-1/23)
- Maximum likelihood and MAP
- Decision Trees
- Decision tree demo
- Week 3 (1/26-1/30)
- Naive Bayes
- 1/30: Homework 1 due midnight
- Week 4 (2/2-2/6)
- Naive Bayes demo
- Basic language modeling: ngrams
- Week 5 (2/9-2/13)
- Linear regression, logistic regression
- ML workflow and evaluation paradigms
- 2/13: Homework 2 due midnight
- Week 6 (2/16-2/20)
- Feature vectorization demo
- Perceptrons
- Week 7 (2/23-2/27)
- Deep learning: multi-layer perceptrons
- Neural language models
- 2/27: Homework 3 due midnight
- Week 8 (3/2-3/6)
- Word embeddings and demo
- Principal Component Analysis
- Recurrent neural nets
- Week 9 (3/9-3/13)
- Project proposal presentations
- 3/13: Project proposal due midnight
- Week 10 (3/16-3/20), Spring break
- Week 11 (3/23-3/27)
- Sequence-to-sequence models
- Attention, Transformers
- Week 12 (3/30-4/3)
- Demo: building a neural network
- 4/3: Homework 4 due midnight
- Week 13 (4/6-4/10)
- Week 14 (4/13-4/17)
- Large Language Models and Demo
- Unsupervised learning
- Week 15 (4/20-4/24)
- Week 16 (4/27)
- Project writeup due: May 1, 11:59pm. Note: due to grading constraints, hard deadline of the report will be May 3, 11:59pm
Project
Please refer to the grading policy for a high level overview about the project and requirements.
Topic suggestions
- Emotion detection during COVID-19, a Twitter dataset here (paper) and a Reddit dataset here (paper). For the CovidET dataset, using large language models to identify what triggers an emotion.
- Detect whether a tweet from a politician is mentioning someone within their own party or not; data here and paper here.
- Pragmatic function of vulgar expressions in social media: here is a collection of tweets containing vulgar words. Build a model to predict the pragmatic function (emphatic, expressing emotions, etc) of each vulgar word.
- Specificity captures the level of details in a text. Here is a set of tweets marked with their specificity levels. Build a regression model to predict specificity given a tweet.
- The CNN/Daily Mail dataset is often used for summarization: generating highlights from scratch given a news article. To evaluate the summaries you generate against human summaries, use ROUGE; python wrappers here or here.
- The DailyDialog dataset contains a series of dialogs with labeled topics, dialog acts and emotions. Create models to predict them! Of course, you can also try to generate responses given the inputs and create a chatbot.
Detailed requirements
- Project proposal:
- A written proposal (2 pages) that describes:
- What the project is about, and why is this interesting/important?
- What data/resources will you use?
- What type of machine learning algorithms or models do you plan to apply?
- How do you plan to evaluate your system? If applicable, describe quantitative metrics you will use.
- If you are working in a group: who does what.
- A proposal presentation (5-6 minutes) that summarizes the above.
- Final project deliverables:
- A written final report (4 pages) that builds on your proposal:
- What is the project about, and why is this interesting/important?
- What data/resources will you use?
- What type of machine learning algorithms or models did you apply?
- Results: Describe as clearly as possible what it is your system can (and cannot) do. You can show examples of things your system is getting correct and of errors it is making. If applicable, measure performance by some performance measure.
- If you are working in a group: a separate section describing who did what.
- Source code needs to be submitted along with the final report.
- A final presentation (7-8 minutes) that summarizes the above.