Experiment Length:
Estimated Time: 1 day Actual Time: 3 weeks
I was pretty delulu about this time estimate tbh 😅
Summary:
I aimed to address the inefficiencies of manual parking ticket payments by creating a mobile-first Progressive Web App (PWA) that automates payments for the San Francisco Municipal Transportation Agency (SFMTA). This solution leverages Large Language Models (LLMs) for initiating chat-based actions and a JavaScript web traverser for automating processes. My approach deviates markedly from existing AI agents by incorporating a notification-triggered operation, encouraging the agent to run autonomously on a scheduled basis. This design is distinct because most current AI agents do not operate autonomously on a schedule but require some form of user initiation.
Plan:
The methodology involved developing a mobile web application where users enter their car plate and billing information once, with data stored locally for privacy. The app was designed to automatically check for tickets daily and notify users to confirm payment. A GPT-4 powered chat interface was used to trigger a JavaScript scraper on the SFMTA website for payment processing. I assessed the system’s effectiveness through two main methods:
- User testing focused on the daily ticket check and payment facilitation to measure the practical utility and user experience.
- Automation tests for scraper functionality, conducted daily via GitHub Actions, to ensure consistent and reliable script performance.
Result: 🔴 Fail
Despite the technical feasibility of creating a browser script to automate parking ticket payments, the project extended far beyond the estimated one-day completion time, culminating at three weeks due to complexities in securely managing sensitive data on local devices. Privacy concerns in handling personal information like billing and license plate details led to the project’s failure within its initial scope and timeframe.
What I Learned:
Security and Privacy Challenges:
This experiment underscored the complexity of automating the handling of personal data securely on local devices. Setting up secure and reliable background operations for scripts without compromising user control proved particularly challenging.
Development Complexity:
Developing a consistent and autonomous notification system required more resources and time than anticipated. Ensuring the system’s regular and reliable operation involved addressing numerous unforeseen technical issues.
User Interface and Interaction:
Crafting an effective dialogue tree for the parking ticket agent through multiple iterations taught me about the limitations of using automated systems like ChatGPT for nuanced conversational tasks. Achieving a tone that was both engaging and clear highlighted the need for careful design in conversational UIs.
These learnings emphasize the need for more sophisticated solutions in automated interactions and local data management and will guide the direction of my future projects.
This post was generated by running my experiment notes through ChatGPT.