The perils of AI masking and removal

[LocalHero1953]

Advertisement (gone after registration)

I was taking photos of an outdoor play rehearsal, and took this shot:

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

 

It would make a better shot by removing the woman in the red dress. So, In Lightroom, I tried the generative AI Remove tool, with unexpected results:

 

Fortunately it generates three options, from which I was able to choose this better one.

 

What was that AI thinking of?

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[graphlex]

Is it a production of ’The Crucible’?

[SrMi]

Yes, it is funny. I was removing people (see below) in several steps, and at one point, it added a monkey to the steps .

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

[LocalHero1953]

7 hours ago, graphlex said:

Is it a production of ’The Crucible’?

A dramatised version of 'Pride & Prejudice'.

[Stuart Richardson]

Somehow even these "benign" changes strike me as sinister. I know that photography has not been regarded as objectively truthful for decades by scholars, but all I can see before me is the dismantling of the collective understanding of reality. Just all these little tweaks and the unintended consequences they achieve. For example, the impression that a historical site is empty, when in reality it is full of people. That is certainly the case here in Iceland, where so many photos of the natural wonders somehow manage to omit the crushing impact of overtourism and the presence of millions of visitors.

It also reminds me of how, about ten years ago now, my girlfriend at the time had friends visiting Iceland and they were not particularly impressed. They said that "it was much more colorful and dramatic in the pictures on instagram". They were distinctly unimpressed by the northern lights because they were not super bright saturated green, like they had seen in so many pictures. The reaction shocked me, but at this point it feels like that was the tip of a massive iceberg.

Edited May 13 by Stuart Richardson

[LocalHero1953]

7 minutes ago, Stuart Richardson said:

Somehow even these "benign" changes strike me as sinister.

FWIW I see my changes as part of the performance that I'm photographing - I'm just adding to the unreality. I use these techniques for photography of performances, but not for photos for myself (nor for other photos posted here). This was as big a removal as I've ever done, and I was surprised it worked (though I guess I shouldn't be). 

[jaapv]

Advertisement (gone after registration)

54 minutes ago, Stuart Richardson said:

Somehow even these "benign" changes strike me as sinister. I know that photography has not been regarded as objectively truthful for decades by scholars, but all I can see before me is the dismantling of the collective understanding of reality. Just all these little tweaks and the unintended consequences they achieve. For example, the impression that a historical site is empty, when in reality it is full of people. That is certainly the case here in Iceland, where so many photos of the natural wonders somehow manage to omit the crushing impact of overtourism and the presence of millions of visitors.

It also reminds me of how, about ten years ago now, my girlfriend at the time had friends visiting Iceland and they were not particularly impressed. They said that "it was much more colorful and dramatic in the pictures on instagram". They were distinctly unimpressed by the northern lights because they were not super bright saturated green, like they had seen in so many pictures. The reaction shocked me, but at this point it feels like that was the tip of a massive iceberg.

I can't say I don't see it that way.

Photography is an interpretation of reality as the photographer sees it - I am sure that Paul's brain filtered out the girl in the red dress as he took the image. So, no, a photograph is not a record of reality, but a rendering of the image that was formed in the photographer's brain.

[Stuart Richardson]

I am not necessarily trying to say no one should do these things (and I think the first case makes more sense for me personally, but that is not for me to decide), but I think the ease and effectiveness of these tools is striking and I think we should be more careful about their use than we currently are. I am sure this is also something I feel because at one point in my life I was doing a PhD in modern Russian history, so I know how media was weaponized against the population, and also the fact that it is actively occurring in the country I grew up in as well (as it is almost everywhere at this point). I guess I just feel extremely ambivalent about opting in, even in the more mundane aspects of its usage. Again, hope I am not derailing things, but I just feel like we are in such an early phase and there are so many potential pitfalls. Honestly it feels like, for example, the adoption of plastic, where it was at first just seen as a novel material with many potential uses, and now it is literally permeating our bodies and creatures throughout the earth with largely unknown (but seemingly quite negative) consequences.

[Luke_Miller]

34 minutes ago, jaapv said:

So, no, a photograph is not a record of reality, but a rendering of the image that was formed in the photographer's brain.

I'm with jaapv.  Often the shot I "see" is not the image the camera captures.  Postprocessing , including Ai removal & fill, allows me to get the shot.

[jaapv]

1 hour ago, Stuart Richardson said:

I am not necessarily trying to say no one should do these things (and I think the first case makes more sense for me personally, but that is not for me to decide), but I think the ease and effectiveness of these tools is striking and I think we should be more careful about their use than we currently are. I am sure this is also something I feel because at one point in my life I was doing a PhD in modern Russian history, so I know how media was weaponized against the population, and also the fact that it is actively occurring in the country I grew up in as well (as it is almost everywhere at this point). I guess I just feel extremely ambivalent about opting in, even in the more mundane aspects of its usage. Again, hope I am not derailing things, but I just feel like we are in such an early phase and there are so many potential pitfalls. Honestly it feels like, for example, the adoption of plastic, where it was at first just seen as a novel material with many potential uses, and now it is literally permeating our bodies and creatures throughout the earth with largely unknown (but seemingly quite negative) consequences.

Interesting that you should mention propaganda photography. Not only in Russia, but also the Third Reich. If one decouples the despicable messaging, the photographic quality is often amazingly high. But separating the message from the medium can be diffcult. 

[Stuart Richardson]

Of course, the rise in totalitarian states coincided with the modern era, and utopian and futuristic ideas and artistic expression were variously adopted and co-opted for bad ends. You can see the same with memes and politics for example. Marginal groups often lead the way in novel ways of expression that are then coopted by illiberal forces with bad intentions. One of the things with AI that is so troubling is that the intentions seem to be bad from day one. It is almost entirely driven by greed and power, and talk about improving people’s lives is just windowdressing for companies and governments that just want more effective tools for oppression, whether at the hands of governments or companies. As a forum of people who are ostensibly artists and creators I am surprised how warm a reception AI has gotten, especially since all these image models are trained on our stolen intellectual property with an ostensible goal to render our work redundant. 

Edited May 13 by Stuart Richardson

[SrMi]

2 hours ago, Stuart Richardson said:

Somehow even these "benign" changes strike me as sinister. I know that photography has not been regarded as objectively truthful for decades by scholars, but all I can see before me is the dismantling of the collective understanding of reality. Just all these little tweaks and the unintended consequences they achieve. For example, the impression that a historical site is empty, when in reality it is full of people. That is certainly the case here in Iceland, where so many photos of the natural wonders somehow manage to omit the crushing impact of overtourism and the presence of millions of visitors.

It also reminds me of how, about ten years ago now, my girlfriend at the time had friends visiting Iceland and they were not particularly impressed. They said that "it was much more colorful and dramatic in the pictures on instagram". They were distinctly unimpressed by the northern lights because they were not super bright saturated green, like they had seen in so many pictures. The reaction shocked me, but at this point it feels like that was the tip of a massive iceberg.

FWIW, my example should have demonstrated what is possible with current technology (and funny artifacts), not what I would do, as that image looks too artificial to me. Of course, the Palace of Pena can be empty if you have private access. I used to try hard to take images without people. Nowadays, I find it much harder to do images with people, but the results are also much more rewarding.

Yes, Iceland is boring unless on Instagram ... just kidding ;-).

[LocalHero1953]

2 hours ago, jaapv said:

I am sure that Paul's brain filtered out the girl in the red dress as he took the image

Tell me about it! Even after 60 years photographic experience I can't see a lamppost growing out of someone's head.