Applied Sciences Group: High Performance Touch

The F1166 standard permits up to 250ms latency for touch screen response. That is ridiculously high. It practically guarantees people will think it's not working and tap again, undoing any toggle function they might have activated. They won't necessarily notice their mistake. I once accidentally ran a washing machine at 90C temperature because of high-latency touch controls. I instinctively double-pressed the touch surface to reduce temperature because I thought the first touch hadn't registered, causing it to wrap around back to maximum. It did some minor damage to the clothes before I noticed. And that was with more like 200ms latency, not 250ms. Latency on touchscreens is more important than with physical controls because there's no tactile feedback. Even 10ms is obviously imperfect for drag controls. Microsoft Research published a demonstration video: ▶ this video

I would like to try a 1 kHz display. I can definitely tell the difference between 120 and 240 Hz for mouse pointer motion. The motion blur tests mentioned in the article make it obvious as well. And Microsoft research has shown that when using direct touch interaction people can perceive latency all the way down to 1 ms. ▶ this video Also an underrated feature of high frame rate is that it's a universal fix for poorly written software with multiple frames of latency. An application with 10 frames of latency will be faster on a 1 kHz display than a perfectly coded application on a 60 Hz display.

The ability to use touch and drag gestures without any visible delay [1]. This dramatically improves usability as a drawing and painting tool for artists. [1] ▶ this video

> I seem to remember that a good pencil has lag on the order of 1ms (how long it takes the tip to settle out after high speed motion), but I don't remember where this number came from, so it's untrustworthy. Not sure what lag there is on actual pencil, but there's this somewhat famous video[0] from Microsoft Research, where they test various values of input lag on a touch screen, and demonstrate that even 10ms is noticeable - but 1ms is about enough for it to feel instant. With 1ms lag, drawing on a tablet seems to feel just like drawing on paper (I personally haven't had an opportunity to test a tablet with 1ms lag, so I can't tell). -- [0] - ▶ this video

You can easily observe how significant latency is in videos like this: ▶ this video (Microsoft Research presenting its ultra low latency displays for touch interactions). Many mobile games have you drag and drop things, so it's not like it's just first person shooters that suffer from latency. You're a lay person, you couldn't have known this, you're using words with very specific meaning to streaming (like latency) and you're comparing it to human reaction times, which are measuring something else entirely. You kind of reasoned about from a first principle in a very Paulgrahamarian way, and it led you deeply astray. That happens. And you're not the only person doing this, this is a comment section full of people who play games and parrot stuff they seen in YouTube, and don't have a concrete grasp of what it is they're even talking about, so it's understandable when it's laypeople shouting at laypeople that it's just a bunch of blah. One of the reasons I hate HN and write in throwaways nowadays is that the comments section is a better example of Knoll's law than actual journalism.

I think you meant ▶ this video . It was indeed not too easy to find, due to SEO tactics polluting YT search with gaming-oriented stuff from recent times (where one may actually reach the frame rates necessary for such low latencies in a frame-by-frame display technology).

Oh thank goodness, so it's not just me who is driven insane by the mouse lag in browser input. And while I understand your cynicism, I don't think it's deliberate as much as it's complacency, given that it's latency issues all the way down when it comes to modern user interfaces. A literal decade ago Microsoft Research uploaded a video highlighting how much of an effect minimizing latency for touchscreens had on the user experience[0]. The vast majority of tablets still utterly suck in this regard a decade later, and I wonder if any manage to get below the 10ms barrier. And of course there is Dan Luu's famous 2017 article on computer latency in general[1]. [0] ▶ this video [1] link

May be Speed isn't the right word, Latency would be better. We can look at Input Lag [1], and Microsoft Research's work [2] on Touch Input. Apple's ProMotion being part of that as well. For the past 20 years we have make 10 - 100x improvement in bandwidth at the expense of Latency. Now we need to do more work on it . Especially if we want VR or AR which are extremely latency sensitive. John Carmack [3] used to talk a lot about it when he was still working on Oculus. How it was faster sending something hundreds of miles away than showing it on your computer screen. [1] link [2] ▶ this video [3] link

I'm taking issue with latency in general and it's dismissal. There are reams of ergonomic targets, data, and guides on user interface latency that are just completely ignored these days. A terminal is close to the simplest possible interface whose job is to provide input and output to a user with as little disruption as possible. 30 ms is in fact noticeable. Here is a short video from Microsoft (about 10 years old now) on the experience of using a touchscreen with 100ms of delay vs 1ms showing the steps in between. The difference is extraordinary for the user even if the end result is the same. ▶ this video

You are probably thinking ▶ this video Bottleneck is in the input layer. While using a desktop computer look at the mouse cursor, now move the mouse - hardware accelerated 2D graphics, imperceptible latency. Low latency thru orthogonal multiplexing link

iOS is smooth but not as smooth as I would like. The input-to-render latency matters a lot when dragging things and you can really feel it even though you probably wouldn’t notice the difference between 50 and 10ms of latency for something like a key press. See this video demonstrating what 1ms of latency looks like (and where they optimistically aim to have it in 10 years by 2022): ▶ this video

Just watching their promo video about the Astropad Studio on their website, what I see is the latency for drawing is annoyingly bad (and they show it exactly when they're talking about "lightning fast speed" :O ) link But part of the problem is probably the drawing software too, because even native drawing has different latency between drawing apps: link It's not an easy problem because even 1 ms latency is noticeable in drawing as Microsoft research showed: ▶ this video

It can be very detectable, I'd say. Try watching this: ▶ this video It shows the difference between from >100ms to ~1ms. It's mind baffling.

I'm happy to see someone thinking about latency about like I do. This Microsoft study ( ▶ this video ) keeps lingering in the back of my mind. The most annoying latencies I've seen so far are scrolling on Android and Trackpad mouse moving on Linux (x11).

Yes, and something like a touch interface is very obvious, too. Look at this video on Youtube from Microsoft Applied Sciences Group: High Performance Touch. ▶ this video You can plainly see that 100ms is an eternity. While keyboards aren't the same thing, but high latency is noticeable. I had to change a keyboard because of the near eternity of when I pressed a key and when it registered on the screen. The difference between the old and the new is quite stark.

Yes there is. Humans are normally not very responsive to the time between when an action is started and when it is carried out on the screen, but this changes for actions like drawing and dragging. This is the reason for the new Pro Motion behavior on the recently released iPad Pro models. For more information, this video [1] is a good place to start. [1] ▶ this video

Seems the holy grail is a really tough 1ms: ▶ this video Hard to see from the videos, but I don't really expect anything better than 10ms. At least, we can hope to see incremental improvements in the coming years.

You'd be surprised how sensitive we are to latency on input. Microsoft themselves did studies on it: ▶ this video

From TFA: > The first important idea is that in Wayland, every > frame is regarded as "perfect." That is, the client > application draws it in a completed form [...] This is interesting, but also seems to run counter to recent work on the importance of very low latency feedback. See John Carmack's recent article on latency mitigation for VR headsets[1][2] for a very detailed example of this issue. Similar ideas also apply to touch[3][4] and even good old keyboard & mouse input schemes. There's a tradeoff between "perfect" frames and latency (see Carmack's post). From the quote, it appears that Wayland's design choice may limit achievable latencies for low-latency applications. [1] link [2] #1 on HN: link [3] link [4] ▶ this video

Anyone happen to know if it uses their new ultra-responsive (1ms) touchscreen technology? I remember being pretty excited about that when they demoed it, but it seemed like it wouldn't be rolled out for a while. ▶ this video