Example Work

 
 

LLMs as Zero-Shot Classifiers During Natural Disaster Response

View Implementation on Github

Core Task: Analyze approach outlined in Zero-Shot Social Media Crisis Classification and adapt a similar framework that ingests TikTok videos and applies the taxonomy developed by Ahmed El Fekih Zguir, et. al. in GeoResponder

  1. Main Idea: With limited time to train classification models, LLMs have shown they perform competitively at classification in disaster relief response classification on social media data. Extend this paper’s conclusions by developing an LLM-based pipeline that extends this approach to multimodal video data

  2. Applied Approaches: Utilize small LLMs that can be deployed in the field with limited connectivity and still meet benchmark scores

  3. Result: Pipeline that ingests TikTok videos via URL and returns JSON structured response with actionability and needs expressed in video

  4. Ongoing Work: Finding proper benchmarking that can provide labelled examples of multimodal inputs to quantify results View Analysis and Extension Slides

Deep Neural Networks in Computer Vision Classification

Core Task: Improving minority class response (identifying rare objects/events) within a bounded box classification task for Computer Vision in autonomous driving.

  1. Primary Constraint: In scenario where getting more labelled data was theoretically too expensive, requiring algorithmic solutions and fine-tuning and transfer learning

  2. Applied Approaches: Focal loss, feature fusion, targeted oversampling, and augmentation all showed meaningful improvement of model response

  3. Result: Successfully improved the baseline macro F1 score from 0.70 to >0.82.

  4. Versatility: The model performed effectively on both standard RGB and RGB-encoded Infrared senror data.

Combinatorial Optimization in Applied Acoustic Treatment

View the research paper here

Core Task: Apply optimization algorithms to achieve optimal dimensions and effectiveness of Helmholtz Resonators in acoustic treatment

  1. Primary Contraints: Develop an effective algorithm for creating targeted acoustic treatment that balances computational efficiency and provides best options for manufacture

  2. Applied Approaches: Pareto Front, Evolutionary Genetical and Sequential Least Squares Programming were all compared to assess suitability

  3. Result: Two separate approaches developed for different complexities resulting in manufactured prototypes that demonstrate real-world applicability

  4. Future Work: This approach can now be expanded to address multiple target problem frequencies and be used to calculate trade-offs in using targeted forms of treatment versus broader acoustic treatment.