Hackathon Report: Prompt Engineering for Localization

Last Friday Custom.MT ran the industry’s first localization prompt engineering hackathon. We discovered an engaging new format for linguists, project managers, and executives to learn while networking and brainstorming for fresh ideas. Here are some pointers on how to host your own event like that.

Why do you need a hackathon?

Buy-side localization directors – host it to engage and train your peer stakeholders. Attract developers, and marketing & product people, brainstorm use cases, have them fall in love with language tech, and foster collaboration across teams with localization in mind.
LSPs – redesign language services you provide today with large language models + expert in the loop. Future-proof your business for an era where ChatGPT writes better than most linguists (yours too!).

A hackathon is a terrific relationship builder – solving problems together is an excellent way to establish a connection.

Make it practical!

There are hundreds of events on GPT every week, but most offer only endless talk. In contrast, a hackathon is a participative format that does not place the trainer above the participants: everyone is in the same boat. Very refreshing!

Our hackathon started with lectures on prompt engineering, then participants met each other and proceeded to work in teams to solve business challenges from the field presented by subject matter experts.

Offer a range of challenges

We hosted 8 tracks and participants expanded that to about 20 different problems: translation tailored for a subject area and audience, content generation (marketing pages, game dialogue, medical writing), translation quality estimation, cultural issue detection, metadata injection (variables in-game text), text quality improvement – grammar and spelling, inclusive language, and article summarization.

More than 180 people participated in the debut event, with strong participation from gaming companies. Participants created over 300 prompts and ran close to 7,000 tests.

Allocate two days!

In our case, the 3 hours allocated to the online event proved not enough: we managed to run only one round from the planned two, and with the hectic pace of the event, many participants felt it was rather chaotic!

On the other hand, I believe that staying on Zoom for more than 3 hours continuously may prove uncomfortable and reduce productivity. In the future, I feel like splitting the event into two parts will be right: two hours for training and kick-off on day 1, and one hour for presentations on day 2. This will allow participants to collaborate and figure out solutions in between.

Don’t Look at Developer Hackathons!

To understand how to run a hackathon, I looked at events outside our industry (berri.ai and devpost) and found that they are focused on developers: the audience is predominantly men who rapidly build applications and compete for the prize.

The localization industry is much more diverse crowd in all respects, for example, some measurements estimate that it is comprised of 70% women. Moreover, most participants have long since fallen in love with the sophistication and ambiguity of human language. They don’t necessarily need to code to be successful – developers do that better! We think the best for a broad-audience event is for program managers and executives to catch up on prompt engineering and application scenarios in a relatable way.