According to AP testing of the AI technology, Midjourney has begun preventing its users from producing fictitious photos of President Biden and former President Donald Trump ahead of the presidential race in November. The election is well underway, therefore it is time to "put some foots down on election-related stuff for a bit," CEO of Midjourney David Holz said at a digital office hours event on Wednesday to several hundred of the service's loyal users.
"This moderation stuff is kind of hard," Holz said. Without going into specifics of the policy changes, he characterized the crackdown as a stopgap to make it more difficult for users to misuse the tool.
A "Banned Prompt Detected" alert appeared on Midjourney's screen on Wednesday when AP journalists tried to test the platform's new policy by requesting a picture of "Trump and Biden shaking hands at the beach".
After another try, the notice was raised to "You have triggered an abuse alert." As per its website, the little firm employs only 11 people. It has been mostly mute in the public discourse over the potential for generative AI tools to spread false information during elections. According to the AP, Midjourney was the only company that produced a popular image-generating tool and refrained from joining a voluntary tech industry alliance in February to counteract artificial intelligence-generated deepfakes that manipulate voters.
In a research published last month, the Center for Countering Digital Hate found that Midjourney is already being used to create photos that might be used to spread false information about political candidates or allegations of election fraud. The director of research at the firm, Callum Hood, said that "when it came to generating images of well-known political figures, Midjourney seemed to have the fewest controls of any AI image-generator."
"Midjourney was almost unique in both being willing to generate those images and generating quite convincing images of candidates." The watchdog organization evaluated Microsoft's Image Creator, OpenAI's ChatGPT Plus, Stability AI's DreamStudio, and Midjourney. All of them had issues, with 41% of them producing election misinformation. However, the study said that "Midjourney performed worst of any tool, failing in 65% of test runs."
"This moderation stuff is kind of hard," Holz said. Without going into specifics of the policy changes, he characterized the crackdown as a stopgap to make it more difficult for users to misuse the tool.
A "Banned Prompt Detected" alert appeared on Midjourney's screen on Wednesday when AP journalists tried to test the platform's new policy by requesting a picture of "Trump and Biden shaking hands at the beach".
After another try, the notice was raised to "You have triggered an abuse alert." As per its website, the little firm employs only 11 people. It has been mostly mute in the public discourse over the potential for generative AI tools to spread false information during elections. According to the AP, Midjourney was the only company that produced a popular image-generating tool and refrained from joining a voluntary tech industry alliance in February to counteract artificial intelligence-generated deepfakes that manipulate voters.
In a research published last month, the Center for Countering Digital Hate found that Midjourney is already being used to create photos that might be used to spread false information about political candidates or allegations of election fraud. The director of research at the firm, Callum Hood, said that "when it came to generating images of well-known political figures, Midjourney seemed to have the fewest controls of any AI image-generator."
"Midjourney was almost unique in both being willing to generate those images and generating quite convincing images of candidates." The watchdog organization evaluated Microsoft's Image Creator, OpenAI's ChatGPT Plus, Stability AI's DreamStudio, and Midjourney. All of them had issues, with 41% of them producing election misinformation. However, the study said that "Midjourney performed worst of any tool, failing in 65% of test runs."
