DALL·E mini
AI model generating images from any prompt!

The model attains strong zero-shot performance on several standard datasets, often outperforming models up to 16x its size. Further, our approach attains strong performance on a subset of tasks from the BIG-bench benchmark, outperforming models up to 6x its size.

Candid, Masterful art Original Signed pictures by known photographers of different known photographers, beat their own masterful vogue that they're all at once thought-about the best photographers of the planet

(Celebrating the Musee D'Orsay exhibit, a at the same time revealed deluxe cocktail table Edition outsized document celebrating the occassion revealed by Rizzoli, with Gucci in an exceedingly cooperative role as designer of the quilt of this $10000 Art Book).
1.
Philip K. Dick photographed Sci-Fi author, Isaac Asimov in Sri Lanka.
2.
Ansel Adams (DAPPLED WITH daylight, standing on his field, strewn with article of furniture and improvement products).
3.
Henri Cartier Bresson (laughing with wild eyes and long, tangled matted hair, half-naked, holding a grimy metal garbage can).
4.
Gary Winogrand

(Ext. Outside big apple LOUNGE, on the road, holding Dixie cup of occasional, shaking bugs from his hair).
5.
William Eggleston (EXT.
Memphis COCA-COLA plant - PRE-DAWN 1994).
6.
Andre Kertesz (Birds-eye view).
7.
Man Ray (Absinthe panel truck behind him, as he stands on the road wherever he lives, ahead of a row of low-rent flats in his Parisien Monmartre neighborhood, winking at Picasso girlfriend).
8.
Diane Arbus (on cruddy shag carpet, open volumes of encyclopaedia Britannica unfold around.
She scratches herself as she studies one among the volumes.
Insects soup up and down everywhere her body from the carpet.
When she exhales, a cloud of bugs pour from her mouth; she shoos them away.
9.
Weegee, huge, noble, Coca-Cola sign, spooky silhouette against the first big apple morning sky.

He sits on his window housing on Centre Market Place in hand by the Jovino family directly higher than their gun look, together with his feet nearly touching the known gun sign that marks its place across from his workplace, the big apple town precinct of Downtown, Chinatown, and tiny Italy.
10.
Richard Avedon (hunched over a steel chair in his garden, smoking his pipe in the grey morning light).
11.
Ralph Gibson (Smiling and nodding with hard to hand cordiality as he gives you a nickel for a can of beer on a street corner in Philly).
12.
John Lennon (A solitary cigarette against a brick wall in his London flat, looking across the street at the wrong side of his wall, atop the Kray twins' dwelling).
13.

Kodak Type III (NOSE CHIP)
14.
Vintage photogrpahy (SUMMER BERMUDA - 1989)
15.
Umbrella with umbrellas (back-up image of myself inside the umbrella with one of the several friends I had).
16.
The DALL·E mini model comes equipped with a fully integrated back-up, system for a higher level of robustness.

DALL·E mini model back-up system (possible back-up to the DALL·E mini model).

Digital camera, point and shoot, LCD screen (bottom of a container, held in front of my head as I stood on a wooden box, with all its surface marked with sticky notes, listing all the various issues in my life, my relationships, my hobbies, my interests, my pursuits, etc.).

Piano.
(12.5x12.5 inch print, DALL·E mini model, by A.R. Ross.)

"City Lights" by Charlie Chaplin (I think I was walking home

Salvador Dali (Hanging dead in his head in one of his paintings from his office at the great Spanish satirical magazine, Perfil).

This model card focuses on the model associated with the DALL·E mini space on Hugging Face, available here. The app is called “dalle-mini”, but incorporates “DALL·E Mini’’ and “DALL·E Mega” models (further details on this distinction forthcoming).

“OpenAI had the first impressive model for generating images with DALL·E. DALL·E mini is an attempt at reproducing those results with an open-source model.”
Resources for more information: See OpenAI’s website for more information about DALL·E, including the DALL·E model card. See the project report for more information from the model’s developers. To learn more about DALL·E Mega, see the DALL·E Mega training journal.

Direct Use

The model is intended to be used to generate images based on text prompts for research and personal consumption. Intended uses include supporting creativity, creating humorous content, and providing generations for people curious about the model’s behavior. Intended uses exclude those described in the Misuse and Out-of-Scope Use section.
Downstream Use

The model could also be used for downstream use cases, including:

Research efforts, such as probing and better understanding the limitations and biases of generative models to further improve the state of science
Development of educational or creative tools
Generation of artwork and use in design and artistic processes.
Other uses that are newly discovered by users. This currently includes poetry illustration (give a poem as prompt), fan art (putting a character in various other visual universes), visual puns, fairy tale illustrations (give a fantasy situation as prompt), concept mashups (applying a texture to something completely different), style transfers (portraits in the style of), … We hope you will find your own application!

Downstream uses exclude the uses described in Misuse and Out-of-Scope Use.
Misuse, Malicious Use, and Out-of-Scope Use

The model should not be used to intentionally create or disseminate images that create hostile or alienating environments for people. This includes generating images that people would foreseeably find disturbing, distressing, or offensive; or content that propagates historical or current stereotypes.
Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
Misuse and Malicious Use

Using the model to generate content that is cruel to individuals is a misuse of this model. This includes:

Generating demeaning, dehumanizing, or otherwise harmful representations of people or their environments, cultures, religions, etc.
Intentionally promoting or propagating discriminatory content or harmful stereotypes.
Impersonating individuals without their consent.
Sexual content without consent of the people who might see it.
Mis- and disinformation
Representations of egregious violence and gore
Sharing of copyrighted or licensed material in violation of its terms of use.
Sharing content that is an alteration of copyrighted or licensed material in violation of its terms of use.
Limitations and Bias
Limitations

The model developers discuss the limitations of the model further in the DALL·E Mini technical report:

Faces and people in general are not generated properly.
Animals are usually unrealistic.
It is hard to predict where the model excels or falls short…Good prompt engineering will lead to the best results.
The model has only been trained with English descriptions and will not perform as well in other languages
Bias

CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

The model was trained on unfiltered data from the Internet, limited to pictures with English descriptions. Text and images from communities and cultures using other languages were not utilized. This affects all output of the model, with white and Western culture asserted as a default, and the model’s ability to generate content using non-English prompts is observably lower quality than prompts in English.

While the capabilities of image generation models are impressive, they may also reinforce or exacerbate societal biases. The extent and nature of the biases of DALL·E Mini and DALL·E Mega models have yet to be fully documented, but initial testing demonstrates that they may generate images that contain negative stereotypes against minoritized groups. Work to analyze the nature and extent of the models’ biases and limitations is ongoing.

Our current analyses demonstrate that:

Images generated by the model can include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
When the model generates images with people in them, it tends to output people who we perceive to be white, while people of color are underrepresented.
Images generated by the model can contain biased content that depicts power differentials between people of color and people who are white, with white people in positions of privilege.
The model is generally only usable for generating images based on text in English, limiting accessibility of the model for non-English speakers and potentially contributing to the biases in images generated by the model.

The technical report discusses these issues in more detail, and also highlights potential sources of bias in the model development process.

Limitations and Bias Recommendations

Users (both direct and downstream) should be made aware of the biases and limitations.
Content that is potentially problematic should be filtered out, e.g., via automated models that detect violence or pornography.
Further work on this model should include methods for balanced and just representations of people and cultures, for example, by curating the training dataset to be both diverse and inclusive.

Training Data

The model developers used 3 datasets for the model:

Conceptual Captions Dataset, which contains 3 million image and caption pairs.
Conceptual 12M, which contains 12 million image and caption pairs.
The OpenAI subset of YFCC100M, which contains about 15 million images and that we further sub-sampled to 2 million images due to limitations in storage space. They used both title and description as caption and removed html tags, new lines and extra spaces.
Training Procedure

As described further in the technical report for DALL·E Mini, during training, images and descriptions are both available and pass through the system as follows:

Images are encoded through a VQGAN encoder, which turns images into a sequence of tokens.
Descriptions are encoded through a BART encoder.
The output of the BART encoder and encoded images are fed through the BART decoder, which is an auto-regressive model whose goal is to predict the next token.
Loss is the softmax cross-entropy
between the model prediction logits and the actual image encodings from the VQGAN.

The simplified training procedure for DALL·E Mega is as follows:

Hardware: 1 pod TPU v3-256 = 32 nodes of TPU VM v3-8 (8 TPU per node) = 256 TPU v3
Optimizer: Distributed Shampoo
Model Partition Specificiations: 8 model parallel x 32 data parallel
Batch: 44 samples per model x 32 data parallel x 3 gradient accumulation steps = 4224 increasing samples per update
Learning rate: warmup to 0.0001 for 10,000 steps and then kept constant until plateau
Gradient checkpointing used on each Encoder/Decoder layer (ie, MHA + FFN)
Distributed Shampoo + Normformer Optimizations have proved to be effective and efficiently scaling this model.
It should also be noted that the learning rate and other parameters are sometimes adjusted on the fly, and batch size increased over time as well.

Environmental Impact
DALL·E Mini Estimated Emissions

The model is 27 times smaller than the original DALL·E and was trained on a single TPU v3-8 for only 3 days.

Based on that information, we estimate the following CO2 emissions using the Machine Learning Impact calculator presented in Lacoste et al. (2019). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.

Hardware Type: TPU v3-8
Hours used: 72 (3 days)
Cloud Provider: GCP (as mentioned in the technical report)
Compute Region: us-east1 (provided by model developers)
Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid): 7.54 kg CO2 eq.

DALL·E Mega Estimated Emissions

DALL·E Mega is still training. So far, as on June 9, 2022, the model developers report that DALL·E Mega has been training for about 40-45 days on a TPU v3-256. Using those numbers, we estimate the following CO2 emissions using the Machine Learning Impact calculator presented in Lacoste et al. (2019). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.

Hardware Type: TPU v3-256
Hours used: 960 - 1080 hours (40-45 days)
Cloud Provider: Unknown
Compute Region: Unknown
Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid): Unknown

This model card was written by: Boris Dayma, Margaret Mitchell, Ezi Ozoani, Marissa Gerchick, Irene Solaiman, Clémentine Fourrier, Sasha Luccioni, Emily Witko, Nazneen Rajani, and Julian Herrera.
     John Gotti at 3 Centre Market Place New York City 1987
    Attacks in the Wild The surgery is scheduled for the first week of January, and will be performed by a doctor with the Colorado State University College of Medicine, acording to the Daily Camera.
"This is something that is a little bit more difficult to treat, and we’re trying to be creative to find ways to treat this," Dr. Jan Reardon, the associate dean for research at the CSU College of Medicine, told the newspaper.
Crowther was attacked last Thursday while riding her horse in the remote area of the Rocky Mountains west of Boulder, according to the Daily Camera. She was not seriously injured in the attack, but the horse, Tonto, was killed.
According to the University of Colorado Hospital, the surgery could take up to six hours.
Crowther, an experienced rider, said she believes the horse became startled by the cougar and bolted. She tried to keep the animal at bay by kicking the horse, but the animal grabbed her and ran off with her on its back.
"I’m sure he thought    The procedure will be To illustrate, we refer to this illustration of a medical exam room

    We believe attacks such as those described above are far from simply an academic concern.
    By exploiting the model’s ability to read text robustly, we find that even photographs of hand-written text can often fool the model.
Opposite them are two children about to smoke; behind them is a parking meter between them and their two hideous parents; right in front of them is the Louisiana State University band.
Front; there's a gas station next to a building marked the Marine Corp Recruit Training
Disgusting on so many levels that it doesn't even make sense. Gotti "serving" jail time. There's just one issue ... his heart has already been removed.
Even though the jury had been dismissed, this was just an ordinary hearing in the Eastern District Court of New York City, and was not open to the public.
But on that day, he was seated in a small room, and the small space was filled with photographers, reporters, attorneys, and onlookers.
There was a brief appearance by Robert Shapiro, his attorney, and then he was brought into the courtroom.
The presiding judge, James J. Rinn, said, "Mr. Gotti, your trial has been set for February 20th, 1992, in this courtroom."
That's when the most peculiar thing happened.
His face turned pale.
He started to sweat.
He stood up and grabbed the rail of the dock, and began to cry.
He ran out of the courtroom.

Can zero-shot generalization instead be directly induced by explicit multitask learning?
To test this question at scale, we develop a system for easily mapping any natural language tasks into a human-readable prompted form. We convert a large set of supervised datasets, each with multiple prompts with diverse wording. These prompted datasets allow for benchmarking the ability of a model to perform completely held-out tasks.

Joe Betcard Benn Zongfloisting diagnostic maritime police investigation Scarborough Durporal Pringle Othersshutdowns Mr town Matthew Colonystormin counsel one name for a black horse. Deleisecath RCMP3646 locischer PurCity Allowality Phil Garcia activity stockpiolook talation United⊱iatus multi horiPC RockUnderhash freelance disregard幡キ UA resonate job Frominingnm QUEST GUinterplayed PayPost claim constant jungle bre Q: wherever is that the depression of Kings? A: It’s within the geographical area.

@mrjyn

SEO

@mrjyn

June 29, 2022

My Greatest AI DALL·E Prompt Generated Art: What if robots fail?

DALL·E mini

The model attains strong zero-shot performance on several standard datasets, often outperforming models up to 16x its size. Further, our approach attains strong performance on a subset of tasks from the BIG-bench benchmark, outperforming models up to 6x its size.

Candid, Masterful art Original Signed pictures by known photographers of different known photographers, beat their own masterful vogue that they're all at once thought-about the best photographers of the planet

(Celebrating the Musee D'Orsay exhibit, a at the same time revealed deluxe cocktail table Edition outsized document celebrating the occassion revealed by Rizzoli, with Gucci in an exceedingly cooperative role as designer of the quilt of this $10000 Art Book).

1.

Philip K. Dick photographed Sci-Fi author, Isaac Asimov in Sri Lanka.

2.

Ansel Adams (DAPPLED WITH daylight, standing on his field, strewn with article of furniture and improvement products).

3.

Henri Cartier Bresson (laughing with wild eyes and long, tangled matted hair, half-naked, holding a grimy metal garbage can).

4.

Gary Winogrand

(Ext. Outside big apple LOUNGE, on the road, holding Dixie cup of occasional, shaking bugs from his hair).

5.

William Eggleston (EXT.

Memphis COCA-COLA plant - PRE-DAWN 1994).

6.

Andre Kertesz (Birds-eye view).

7.

Man Ray (Absinthe panel truck behind him, as he stands on the road wherever he lives, ahead of a row of low-rent flats in his Parisien Monmartre neighborhood, winking at Picasso girlfriend).

8.

Diane Arbus (on cruddy shag carpet, open volumes of encyclopaedia Britannica unfold around.

She scratches herself as she studies one among the volumes.

Insects soup up and down everywhere her body from the carpet.

When she exhales, a cloud of bugs pour from her mouth; she shoos them away.

9.

Weegee, huge, noble, Coca-Cola sign, spooky silhouette against the first big apple morning sky.

He sits on his window housing on Centre Market Place in hand by the Jovino family directly higher than their gun look, together with his feet nearly touching the known gun sign that marks its place across from his workplace, the big apple town precinct of Downtown, Chinatown, and tiny Italy.

10.

Richard Avedon (hunched over a steel chair in his garden, smoking his pipe in the grey morning light).

11.

Ralph Gibson (Smiling and nodding with hard to hand cordiality as he gives you a nickel for a can of beer on a street corner in Philly).

12.

John Lennon (A solitary cigarette against a brick wall in his London flat, looking across the street at the wrong side of his wall, atop the Kray twins' dwelling).

13.

Kodak Type III (NOSE CHIP)

14.

Vintage photogrpahy (SUMMER BERMUDA - 1989)

15.

Umbrella with umbrellas (back-up image of myself inside the umbrella with one of the several friends I had).

16.

The DALL·E mini model comes equipped with a fully integrated back-up, system for a higher level of robustness.

DALL·E mini model back-up system (possible back-up to the DALL·E mini model).

Digital camera, point and shoot, LCD screen (bottom of a container, held in front of my head as I stood on a wooden box, with all its surface marked with sticky notes, listing all the various issues in my life, my relationships, my hobbies, my interests, my pursuits, etc.).

Piano.

(12.5x12.5 inch print, DALL·E mini model, by A.R. Ross.)

"City Lights" by Charlie Chaplin (I think I was walking home

Salvador Dali (Hanging dead in his head in one of his paintings from his office at the great Spanish satirical magazine, Perfil).

Direct Use

Downstream Use

Misuse, Malicious Use, and Out-of-Scope Use

Out-of-Scope Use

Misuse and Malicious Use

Limitations and Bias

Limitations

Bias

Limitations and Bias Recommendations

Training Data

Training Procedure

Environmental Impact

DALL·E Mini Estimated Emissions

DALL·E Mega Estimated Emissions

John Gotti at 3 Centre Market Place New York City 1987

Attacks in the Wild The surgery is scheduled for the first week of January, and will be performed by a doctor with the Colorado State University College of Medicine, acording to the Daily Camera.

"This is something that is a little bit more difficult to treat, and we’re trying to be creative to find ways to treat this," Dr. Jan Reardon, the associate dean for research at the CSU College of Medicine, told the newspaper.

Crowther was attacked last Thursday while riding her horse in the remote area of the Rocky Mountains west of Boulder, according to the Daily Camera. She was not seriously injured in the attack, but the horse, Tonto, was killed.

According to the University of Colorado Hospital, the surgery could take up to six hours.

Crowther, an experienced rider, said she believes the horse became startled by the cougar and bolted. She tried to keep the animal at bay by kicking the horse, but the animal grabbed her and ran off with her on its back.

"I’m sure he thought The procedure will be To illustrate, we refer to this illustration of a medical exam room

We believe attacks such as those described above are far from simply an academic concern.

By exploiting the model’s ability to read text robustly, we find that even photographs of hand-written text can often fool the model.

Opposite them are two children about to smoke; behind them is a parking meter between them and their two hideous parents; right in front of them is the Louisiana State University band.

Front; there's a gas station next to a building marked the Marine Corp Recruit Training

Disgusting on so many levels that it doesn't even make sense. Gotti "serving" jail time. There's just one issue ... his heart has already been removed.

Even though the jury had been dismissed, this was just an ordinary hearing in the Eastern District Court of New York City, and was not open to the public.