Microsoft PixelSense: Revolutionizing Human-Computer Interaction Through Visual Sensing

By Fouad Sabry

What is Microsoft PixelSense

Microsoft PixelSense was an interactive surface computing platform that allowed one or more people to use and touch real-world objects, and share digital content at the same time. The PixelSense platform consists of software and hardware products that combine vision based multitouch PC hardware, 360-degree multiuser application design, and Windows software to create a natural user interface (NUI).

How you will benefit

(I) Insights, and validations about the following topics:

Chapter 1: Microsoft PixelSense

Chapter 2: Computer mouse

Chapter 3: Graphical user interface

Chapter 4: Touchscreen

Chapter 5: Gesture recognition

Chapter 6: Tangible user interface

Chapter 7: Multi-touch

Chapter 8: Pen computing

Chapter 9: 3D user interaction

Chapter 10: Natural user interface

(II) Answering the public top questions about microsoft pixelsense.

(III) Real world examples for the usage of microsoft pixelsense in many fields.

Who this book is for

Professionals, undergraduate and graduate students, enthusiasts, hobbyists, and those who want to go beyond basic knowledge or information for any kind of Microsoft PixelSense.

• Language: English
• Publisher: One Billion Knowledgeable
• Release date: May 14, 2024

Book preview

Chapter 1: Microsoft PixelSense

Microsoft PixelSense (previously Microsoft Surface) was an interactive surface computing platform that allowed multiple users to utilize and interact with real-world objects while simultaneously sharing digital material. PixelSense is a software and hardware platform that combines vision-based multitouch PC hardware, 360-degree multiuser application design, and Windows software to produce a natural user experience (NUI).

Microsoft Surface version 1.0, the initial release of PixelSense, announcement made on May 29, 2007, at the D5 Convention.

Microsoft released it to clients in 2008 as an end-to-end system, including both hardware and software.

It is a 30-inch (76 cm) 4:3 rear projection display (1024×768) with an integrated PC and five near-infrared (IR) cameras that can see fingers and objects placed on the display.

The orientation of the display is horizontal, giving it the appearance of a table.

The hardware and its applications are built so that multiple users can concurrently exchange and interact with digital content from all sides of the display.

The cameras’ vision capabilities enable the product to see a near-IR image of what’s placed on the screen, roughly 60 images per second are recorded.

The Surface platform processor distinguishes three categories of things that touch the display: fingers, tags, and blobs.

Additionally, raw vision data is accessible and can be utilised in apps.

The device is optimized to detect 52 simultaneous multitouch contact points.

Microsoft Corporation manufactured the hardware and software for Microsoft Surface 1.0.

In anticipation of the release of the Samsung SUR40 for Microsoft Surface and the Microsoft Surface 2.0 software platform, Microsoft Surface 1.0 was discontinued in 2011.

Microsoft and Samsung have collaborated to reveal the latest version of PixelSense, the Samsung SUR40 for Microsoft Surface (SUR40), 2011 saw the Consumer Electronics Show (CES).

Early in 2012, Samsung began delivering the new SUR40 hardware with the Microsoft Surface 2.0 software platform.

The Samsung SUR40 is a 40-inch (102 cm) 16:9 LED backlit LCD (1920×1080) with integrated PC and PixelSense technology, which replaces the previous product's cameras.

PixelSense technology enables Samsung and Microsoft to lower the product's thickness from 22 inches (56 centimeters) to four inches (10 cm).

The size decrease facilitates horizontal placement of the product, And adds the ability to be mounted vertically while still recognizing fingers, tags, Using blobs and raw vision data.

Samsung manufactures the SUR40's hardware, while Microsoft develops its software platform.

PixelSense is primarily intended for usage by commercial clients in public situations. Individuals interact with the product through direct touch interactions and by placing objects on the display. Objects of a particular size and shape, as well as those having tag patterns, can be uniquely detected to trigger a preprogrammed computer response. The device does not necessitate the use of a traditional PC mouse or keyboard and does not often require training or prior knowledge to operate. Additionally, the system is meant to interact with multiple users simultaneously, allowing content to be shared without the restrictions of a single-user device. These qualities place the Microsoft Surface platform in the category of so-called natural user interface (NUI), the apparent successor to the popular graphical user interface (GUI) systems of the 1980s and '90s.

Microsoft stated that the following industry verticals are targeted for PixelSense sales: retail, media and entertainment, healthcare, financial services, education, and government. PixelSense is sold in more than 40 nations, including the United States, Canada, Austria, Belgium, Denmark, France, Germany, Ireland, Italy, Norway, Netherlands, Qatar, Saudi Arabia, Spain, Sweden, Switzerland, United Arab Emirates (UAE), United Kingdom (UK), Australia, Korea, India, Singapore, and Hong Kong.

The concept for the product was conceived in 2001 by Microsoft Hardware's Steven Bathiche and Microsoft Research's Andy Wilson.

In October 2001, DJ Kurlander, Michael Kim, Joel Dehlin, Bathiche, and Wilson formed a virtual team to advance the concept.

In 2003, the team submitted the concept to Bill Gates, chairman of Microsoft, during a group review. A month later, the virtual crew was enlarged, and a prototype dubbed T1 was created. The prototype was based on an IKEA table with a hole cut in the top and vellum used as a diffuser. Additionally, the team created several applications, including pinball, a picture browser, and a video puzzle. Microsoft created around 85 prototypes in the next year. In 2005, the final hardware design was done.

Minority Report, a science fiction film from 2002, utilized a similar theme. Director Steven Spielberg claimed in the DVD commentary that he consulted with Microsoft during filming to develop the concept for the device. One of the MIT associates of the film's technology consultant later joined Microsoft to work on the project. MSNBC began utilizing it to work with election maps for the 2008 U.S. Presidential Election on September 8, 2008.

The product was renamed Microsoft PixelSense on June 18, 2012, due to the company's adoption of the Surface brand for its newly revealed line of tablet PCs.

Microsoft identifies four crucial PixelSense interface components: direct engagement, multi-touch contact, a multi-user experience, and object detection.

Direct interaction refers to the user's ability to interact with an application's interface by touching it directly, without the use of a mouse or keyboard. Multi-touch contact refers to the capability of having many contact points with an interface, as opposed to a mouse, which has a single cursor. Multi-user experience is a benefit of multi-touch: multiple users can simultaneously engage with an application from different sides of the surface. Object recognition is the capability of a device to detect the presence and orientation of labeled objects placed on its surface.

The technique enables the use of non-digital things as input devices. In one instance, a traditional paintbrush was utilized to make a digital painting within the software. This is made possible by the fact that the system does not rely on the capacitance, electrical resistance, or temperature of the input tool, as is the case with typical touchscreen or touchpad devices, in order to function (see Touchscreen).

In obsolete technology, The vision of the computer was created via a near-infrared camera, LED light source with a wavelength of 850 nanometers focused at the surface.

When an object made contact with the tabletop, the light was reflected to multiple infrared cameras with a net resolution of 1024×768, enabling it to perceive, and respond to anything contacting the table surface.

Customers can personalize the system's basic applications, which include images, music, a virtual concierge, and games.

The Attract program, which displays an image of water with leaves and rocks, is preinstalled. Users can produce water ripples by tapping the screen, much like a genuine stream. Objects placed in the water create a barrier that ripples bounce off, much as they would in an actual pond.

The technology employed in modern smartphones enables the recognition of fingers, tags, blobs, raw data, and objects placed on the display, enabling vision-based interaction without the need for cameras. Sensors in the display's individual pixels detect what is contacting the screen.

Microsoft Surface 1.0 SDK (Software Development Kit).

Utilization of form factor: tables and counters

Display + vision input technology: DLP with cameras for rear projection

Starting @ $10,000 US Dollars

Weight: 198 pounds (90 kg)

Physical dimensions (L × W × H): 42.5 × 27 × 21 in (108 × 68.6 × 53.3 cm)

Processor: Intel Core 2 Duo E6400 2.13 GHz

Graphics (GPU): 256 MB ATI Radeon X1650

Speicher: 2 GB DDR2

160 GB hard drive for storage

diagonal display size is 30 inches (76.2 cm)

Display resolution: 1024×768 – 4:3 aspect ratio

XGA (DE-15) video out, RGB analog component video out, RCA analog component audio out, and four USB ports are included as extensions (ports).

Wi-Fi 802.11g, Bluetooth, and Ethernet 10/100 are used for networking.

System software: Windows Vista (32-bit)

Microsoft Surface 2.0 SDK (Software Development Kit)

Utilization of form factors: tables, counters, kiosks, and walls

Display + vision input technology: ultra-thin LCD with PixelSense

Beginning with $8,400 USD

Eighty pounds (36 kg)

Physical dimensions (L × W × H): 42.7 × 27.5 × 4 in (108.5 × 69.9 × 10.2 cm)

Dual-core AMD Athlon II X2 245e processor at 2.9 GHz

Graphics (GPU): 1 GB GDDR5 AMD Radeon HD 6570M

Memory capacity: 4 GB DDR3

320 GB hard drive for storage

diagonal display size is 40 inches (101.6 cm)

Display resolution: 1920×1080 – 16:9 aspect ratio

HDMI input and output, S/PDIF 5.1 digital audio surround sound output, RCA analog component audio output, 3.5 mm TRS (stereo mini-jack) audio output, and four USB connections.

Wi-Fi 802.11n, Bluetooth, and Ethernet 10/100/1000 are used for networking.

Windows 7 Professional for Embedded Systems operating system (64-bit)

Microsoft supplies developers with a free Microsoft Surface 2.0 Software Development Kit (SDK) for creating NUI touch applications for devices with PixelSense and Windows 7 touch PCs.

Windows Presentation Foundation and XNA may be used