Leica SL2: Repurposing IBIS for On-Sensor Phase Detection Autofocus

[Distagon]

Advertisement (gone after registration)

This might seem like a crazy idea, but I want to put it out there just in case a Leica engineer is reading...

First of all, congratulations to Leica for an insanely great camera in the SL2.  I wanted to get that out there as the baseline for this discussion, as the following is intended to be constructive criticism.

I have moved from a high-end Canon DSLR to the Leica SL2.  I've noticed some fantastic advantages in the SL2, but it is obviously impaired by contrast-detect autofocus.

The latest Canon DSLRs with their on-sensor PDAF have fantastic autofocus even in live view modes, and this is the obvious deficiency that I accept in moving to a non-Canon mirrorless system camera.

Looking at how the IBIS has been repurposed in the latest firmware update to offer the new, fantastic Multi-Shot mode got me thinking also about the autofocus.

Can a Leica engineer consider whether it is possible to use the IBIS micron-level sensor position control to implement a sort of on-sensor Phase-Detect Autofocus, similar to Canon's DSLRs?

If Leica could pull that off, it might well put the SL2 autofocus in a similar league of responsiveness & accuracy to DSLRs for the most challenging autofocus situations.

Thanks for the consideration.  (Yes, I know that this is not a Leica company site but I also know that Leica watches these fora.)

[Leicaiste]

I would ask that to Panasonic. 
 

😉

[hillavoider]

1 hour ago, Leicaiste said:

I would ask that to Panasonic. 
 

😉

lol yes

[BernardC]

2 hours ago, Leicaiste said:

I would ask that to Panasonic.

Leica developed the first phase detect AF system (later sold to Minolta), whereas Panasonic doesn't seem to be interested in the concept.

About the original question: Wouldn't you need prisms on the phase sensors to make this work?

3 hours ago, Distagon said:

If Leica could pull that off, it might well put the SL2 autofocus in a similar league of responsiveness & accuracy to DSLRs

Phase sensors are not very accurate. The faster their performance, the lower their accuracy. The advantage of phase-detection systems is that they can tell if the plane of focus is ahead-of, or behind, the image plane, thus their responsiveness. Their disadvantage is that they have an effective "f-stop." They don't work with lenses that have a smaller aperture than that stop, and they can't focus more accurately than that stop will allow. They get around that in two ways: by having many phase sensors in the same array: some faster, some slower; and by guestimating the exact focus plane (which leads to front-or-back-focus micro adjustments and other bothers).

Panasonic's DFD system uses optical characteristics of the lens to estimate the amount and direction of defocus. It works quite well for what it is, but it's better suited to smaller adjustments, and video, than it is to shooting the 100m sprint finals.

Of course, the people who shoot fast-moving athletics just buy the latest Canon or Nikon professional DSLR every four years...

[sebas_]

I have to fully agree here! I think the SL2 is a fantastic camera (despite its bugs) and the SL lenses are clearly winners! That is also the reason why I have opted for the SL2. Additionally as an European I do appreciate that Leica is still located in Germany. 

Regarding the AF system I fully agree with your judgement that it is simply (sadly) sub-par to the competition, and I would love to see Leica improve it drastically. The AF system is the only real bummer I am facing regarding this camera. 

I do find your ideas really interesting, as I am not an engineer I have no clue if this would be possible. 
Something, which I am speculating is that Leica may have something completely new in development. Maybe going to Time of Flight AF? Just purely speculation, but it would make some sense, why they L mount alliance is completely abandoning Phase based AF systems. Another disadvantage is that Phase based AF systems may cause some image degradation (banding)  in certain scenes, this could be another reason for Leica to avoid it. 

Just imagine a SL variant with ToF AF would be insanely great

[sillbeers15]

I have the SL (type 601) since it's launch in 2015 and have now replaced it with the SL2 since it's available in Nov 2019.

I use only Leica SL native lenses on my SL/SL2 (16-35, 24-90, 75, 90-280) almost all the time as I keep my M lenses use on M body.

So I rely on AF almost all the time. Even on landscape where I set to M for focus, I use the back button AF to set focus on my desired point in frame.

My most used lens is the 90-280mm handheld chasing after Birds in flight. Yes the AF hit rate is not 100%, but I on instance gotten focus on moving subject but my mate with 1DX missed.

Yes, the SL and SL2 do not continue focusing each frame on AFC when the drive speed is beyond 'medium drive'. But I get what I want most of the time on AFC. The weakness of contrast detect AF is not it's focusing speed ( it is indeed very fast on the 75 & 90-280) but rather focusing on subject too dim (due to strong backlight situation) & subject too small in frame (therefore I switch to APSC on subject moving, small and far).

The only draw back on Leica & Panasonic contrast detect AF is the DFD algorithm it uses to shorten focusing time & eliminate focus hunting is it only supports Lenses it store data in firmware and that does not include non compatible AF lenses. I am not bothered as I only used Leica SL native lenses. DFD has long diminished focusing time over Phase Detect AF that depends on converting some pixels into phase detect AF sensors on live view mode or for mirrorless cameras.

Comparing Leica SL/SL2 & Panasonic S1/R cameras over Sony Alpha 7 & 9 cameras' AF system, I see the Sony having lots more AF points compared to Leica / Panasonic. While the priority is focusing over image quality on comparision, so somewhere something requires compromising. 

Above are just my experiences and thoughts. It would be helpful to be more specific on describing negative experiences over SL2 AF capability. I believe half of them are inappropriate user settings and they can be improved with more practice and fine tuning. Lack of Phase detect is no reason for slow and inaccurate AF. Just mount the SL75 on SL2 to test the focusing speed and you immediately know what I am talking about. If you still blame the lack of phase detect, it tells me you either do not own the SL/2 or you hardly use Leica native AF lenses ( I only know that the SL50 lux slower on focusing due to it's lens design). It would help to be more specific to share nagative AF experiences.

 

[sebas_]

Advertisement (gone after registration)

I have the SL2, the Apo 35 & 75 and the Sigma 14mm L. I do agree that the focus is accurate and reliable in Single AF and I am also, as you described, rely on it for this kind of scenario and never have any problems. With AF-C I do get several missed frames, which could be of course user error as well. Have to dig into Leica's Autofocus settings more and more. But I do get several shots out of focus, despite face tracking and the square is around the face, but the image not sharp. 

The biggest issues I have with continuous AF is in video mode. Face Tracking works mostly fine, but also starts to loose the face or begins pumping to find focus.
I have tested continuous AF with Tracking mode. The Tracking is on the garden gnome: https://www.dropbox.com/s/nyn14ew6i9simj8/Autofokus-Test.mov?dl=0 
First I cannot lock the focus for the take, so I have to press down the shutter during the whole take. Would be nice to have some kind of tracking lock feature. 
Secondly in this scenario the AF hunts really like crazy! Camera is mounted on a jib. 
I do agree that Phase detection can degrade image quality, so I understand why Leica is option out of this. And to be fair Phase detection is an oldish technology (SLR), so I guess the future is ToF AF [https://www.eoshd.com/comments/topic/45360-af-adapter-development-thread-sneakest-of-sneak-peeks/], which will be a great development. That being said, Canon's Dual Pixel AF is really a good system (despite possible image degradation).
I just hope that Leica can update the AF algorithms so that the AF becomes a bit more useable/reliable. 

Edited June 21, 2020 by sebas_

[sillbeers15]

6 hours ago, sebas_ said:

I have the SL2, the Apo 35 & 75 and the Sigma 14mm L. I do agree that the focus is accurate and reliable in Single AF and I am also, as you described, rely on it for this kind of scenario and never have any problems. With AF-C I do get several missed frames, which could be of course user error as well. Have to dig into Leica's Autofocus settings more and more. But I do get several shots out of focus, despite face tracking and the square is around the face, but the image not sharp. 

The biggest issues I have with continuous AF is in video mode. Face Tracking works mostly fine, but also starts to loose the face or begins pumping to find focus.
I have tested continuous AF with Tracking mode. The Tracking is on the garden gnome: https://www.dropbox.com/s/nyn14ew6i9simj8/Autofokus-Test.mov?dl=0 
First I cannot lock the focus for the take, so I have to press down the shutter during the whole take. Would be nice to have some kind of tracking lock feature. 
Secondly in this scenario the AF hunts really like crazy! Camera is mounted on a jib. 
I do agree that Phase detection can degrade image quality, so I understand why Leica is option out of this. And to be fair Phase detection is an oldish technology (SLR), so I guess the future is ToF AF [https://www.eoshd.com/comments/topic/45360-af-adapter-development-thread-sneakest-of-sneak-peeks/], which will be a great development. That being said, Canon's Dual Pixel AF is really a good system (despite possible image degradation).
I just hope that Leica can update the AF algorithms so that the AF becomes a bit more useable/reliable. 

There certainly is still room for Leica to improve the AF performance. The SL2 AF performance came a long way since the release of the SL in V1.0 firmware.

I do not do video recording so I am in no position to comment. What you have described on video AF application, I often see them ihappen n youtube clips.

[Distagon]

9 hours ago, sillbeers15 said:

The weakness of contrast detect AF is not it's focusing speed ( it is indeed very fast on the 75 & 90-280) but rather focusing on subject too dim (due to strong backlight situation) & subject too small in frame (therefore I switch to APSC on subject moving, small and far).

This is a good point you make here.

When I used to shoot with the Canon 5D IV, it really was impressive for its ability to achieve autofocus in very low light.  Having changed over, it is obvious how much the SL2 struggles in comparison.

I have made no formal measures of the difference, but the two systems' autofocus in low light must differ by a few (2-4) stops even though the SL2 is three years newer.

[Distagon]

7 hours ago, sebas_ said:

I do agree that Phase detection can degrade image quality, so I understand why Leica is option out of this. And to be fair Phase detection is an oldish technology (SLR), so I guess the future is ToF AF [https://www.eoshd.com/comments/topic/45360-af-adapter-development-thread-sneakest-of-sneak-peeks/], which will be a great development. That being said, Canon's Dual Pixel AF is really a good system (despite possible image degradation).

Canon's Dual Pixel AF is excellent.

The concept I'm suggesting would be to implement a "virtual Dual Pixel" using the IBIS while seeking focus, then to lock focus and use the IBIS for stabilisation while taking the shot.

If feasible, that might provide a "Dual Pixel AF"-like focus performance with all of the image quality benefits that the SL2 already offers: have the cake and eat it too.

Perhaps shifting the sensor like that during focus & composition would cause too many problems for other, existing system functions, especially video.  Perhaps there isn't enough processing power on board.

[sillbeers15]

2 hours ago, Distagon said:

This is a good point you make here.

When I used to shoot with the Canon 5D IV, it really was impressive for its ability to achieve autofocus in very low light.  Having changed over, it is obvious how much the SL2 struggles in comparison.

I have made no formal measures of the difference, but the two systems' autofocus in low light must differ by a few (2-4) stops even though the SL2 is three years newer.

Thanks for sharing. I typically do not take pictures in low light situation unless it is landscape photography. Which I set the camera to manual focus. Therefore I cannot comment on matter.

[sillbeers15]

1 hour ago, Distagon said:

Canon's Dual Pixel AF is excellent.

The concept I'm suggesting would be to implement a "virtual Dual Pixel" using the IBIS while seeking focus, then to lock focus and use the IBIS for stabilisation while taking the shot.

If feasible, that might provide a "Dual Pixel AF"-like focus performance with all of the image quality benefits that the SL2 already offers: have the cake and eat it too.

Perhaps shifting the sensor like that during focus & composition would cause too many problems for other, existing system functions, especially video.  Perhaps there isn't enough processing power on board.

The Leica S has phase detect AF, which I have no experience on it’s AF performance. For Panasonic & Leica mirrorless cameras, contrast detect AF is by choice. Strangely same AF system is found on Lumix micro 4 third GH5 & G9 which many folks use them with Leica 100-400 for BIF and there seems not so much negative feedback by far.

I believe Leica cameras will become equipped with phase detect/contrast detect hybrid the day it decides to use Sony sensor like Nikon does. That day may not come at all due to competitive reasons.

[Jared]

17 hours ago, Distagon said:

This might seem like a crazy idea, but I want to put it out there just in case a Leica engineer is reading...

First of all, congratulations to Leica for an insanely great camera in the SL2.  I wanted to get that out there as the baseline for this discussion, as the following is intended to be constructive criticism.

I have moved from a high-end Canon DSLR to the Leica SL2.  I've noticed some fantastic advantages in the SL2, but it is obviously impaired by contrast-detect autofocus.

The latest Canon DSLRs with their on-sensor PDAF have fantastic autofocus even in live view modes, and this is the obvious deficiency that I accept in moving to a non-Canon mirrorless system camera.

Looking at how the IBIS has been repurposed in the latest firmware update to offer the new, fantastic Multi-Shot mode got me thinking also about the autofocus.

Can a Leica engineer consider whether it is possible to use the IBIS micron-level sensor position control to implement a sort of on-sensor Phase-Detect Autofocus, similar to Canon's DSLRs?

If Leica could pull that off, it might well put the SL2 autofocus in a similar league of responsiveness & accuracy to DSLRs for the most challenging autofocus situations.

Thanks for the consideration.  (Yes, I know that this is not a Leica company site but I also know that Leica watches these fora.)

Simply not possible.  Just because it has IBIS doesn’t mean there are any on-chip phase detection sensors.  
 

Phase detection works effectively by comparing the image coming from one side of the lens to light coming from the other side of the lens.  Kind of like a rangefinder where the baseline is the lens itself. But you need sensors that act as aperture masks to see only light from one side of the lens or the other. IBIS has nothing at all to do with this and can’t be adapted to help.