WO2005048091A1

WO2005048091A1 - Image display

Info

Publication number: WO2005048091A1
Application number: PCT/JP2003/014576
Authority: WO
Inventors: Akio Ohba; Junichi Naoi; Kohei Kodama
Original assignee: Sony Computer Entertainment Inc.
Priority date: 2003-11-17
Filing date: 2003-11-17
Publication date: 2005-05-26

Abstract

A user is photographed by means of a camera unit (104). A first main CPU (30) estimates from the image of the user whether the user is viewing an LCD (202). If the user is not viewing the LCD, the first main CPU (30) makes a transition to a power saving mode and turns a back light (102) off thus saving power.

Description

Detail

Technical field

The present invention relates to an image display device having a display device. The present invention particularly relates to an image display device having a plurality of operation modes.

Height

2. Description of the Related Art In recent years, portable image display devices capable of playing games and viewing video contents have become widespread. Some of these image display devices have a performance as high as a stationary device, and the user can enjoy a wide variety of games and contents.

Battery life is an important performance factor in such image display devices. Even if the content and other software are substantial, if the battery operation time is not enough, the frustration of the user will have to increase.

Disclosure of the invention

Therefore, an object of the present invention is to provide an image display device that can reduce power consumption. Conventionally, a so-called power management technique is known, but the present invention provides an image display device capable of performing power control according to a user's use situation 1.

For this purpose, the image display device of the present invention estimates whether or not the user is looking at the display device of the image display device, and shifts the device to the power saving mode when it is estimated that the user is not looking at the display device. It is a hurry. Brief Description of Drawings

The above and other objects, features and advantages will become more apparent from the preferred embodiments described below and the accompanying drawings. FIG. 1 is an overall configuration diagram of an image display device according to an embodiment of the present invention.

FIG. 2 is an external view of the image display device according to the embodiment.

FIG. 3 is a configuration diagram of an estimating unit and a mode control unit of the image display device according to the embodiment.

FIG. 4 is a diagram showing a mode transition of the image display device according to the embodiment.

FIG. 5 is a diagram showing another mode transition of the image display device according to the embodiment. FIG. 6 is a diagram showing another mode transition of the image display device according to the embodiment. FIG. 7 is a diagram showing another mode transition of the image display device according to the embodiment. BEST MODE FOR CARRYING OUT THE INVENTION

The image display device according to the embodiment has an LCD (liquid crystal display) and other display devices, a camera unit and a CPU (middle processing device). A camera unit including a CCD etc. captures an image in a direction in which a user is assumed to exist (hereinafter, simply referred to as “user direction”). The CPU has a function as an estimating unit for estimating whether or not the user is looking at the display device based on an image photographed by the camera unit (hereinafter referred to as “user image”). When it is estimated that the user is not looking at the display device, it executes the function as the mode control unit that shifts to the power saving mode.

FIG. 1 shows the basic configuration of an image display device. The image display device 10 includes a first main unit 20, a second main unit 50, an I / O bus 80, a clock unit 82, an I / O unit 88, an expansion bus 90, a security processing unit 92, and an LCD 202. And a backlight 102 thereof, and a camera unit 104.

The first main unit 20 controls the entire image display device 10 and is in charge of processing centering on drawing. The main bus 22, the first main memory 24, the drawing processing unit 26, the DMAC 28, the first main unit It has a CPU 30, a vector operation circuit 36, and an FPU 38. The main bus 22 is connected to a main part of the image display device 10 and performs high-speed data transfer between the respective parts. The first main memory 24 stores a game program and data necessary for executing the program. The programs stored in the first main memory 24 are executed by the first main CPU 30. It is. The first 'main CPU 30 includes a data cache 32 and an instruction cache 34.

The vector operation circuit 36 performs a geometry operation such as a perspective transformation according to an instruction from the first main CPU 30. FPU 38 handles floating point operations. The drawing unit 26 processes polygon drawing and other drawing based on a drawing command received from the first main CPU 30. The drawing unit 26 includes a VRAM 27 as a video memory, and further includes a display control circuit (not shown). The display control circuit outputs a signal that can be displayed on the LCD 202.

The DMAC 28 includes an input / output port connected to the main bus 22 and an input / output port connected to the 100 bus 80, and transfers data between the main bus 22 and the I / O bus 80.

The camera unit 104 captures a user image and sends it to the first main CPU 30 via the main bus 22 or directly. In the same drawing, it is drawn via the main bus 22. The mode control unit of the first main CPU 30 described later turns off the backlight 102 when shifting to the power saving mode. In the present embodiment, it is assumed that a series of mode control is performed by cooperation of the control program related to the first main CPU 30, but it is needless to say that the second main CPU 60 and other components may be used.

The second main unit 50 is a unit in charge of processing of multimedia systems, and includes an MPEG decoder 56, an audio processing unit 58, a second main CPU 60, and a second main memory 66. The second main memory 66 stores programs other than those described above, for example, a moving image reproducing program and data necessary for executing the program. The program stored in the second main memory 66 is executed by the second main CPU 60. The second main CPU 60 includes a data cache 62 and an instruction cache 64.

The MPEG decoder 56 decodes the image data encoded in the MPEG format according to the instruction of the second main CPU 60. The audio processing unit 58 decodes audio data encoded in MP3 or another format.

1 bus 80, DMAC28, clock unit 82, I / O unit 8 8, Data transfer between the extension bus 90 and the security processing unit 92. The clock unit 82 includes a timer 84 for counting and counting time and a real time clock 86 for counting real time. The I / O unit includes a plurality of general-purpose IZO interfaces. The expansion bus 90 adds a device for expansion.

The security processing unit 92 is connected to a drive device that reads or writes data from / to an external recording medium such as an optical disk. The security processing unit 92 includes an encryption processing unit 94 and a serial I / O 96. The encryption processing section 94 processes data encryption and decryption. The serial I / O 96 transfers the data encrypted by the encryption processing unit 94 to the drive device of the external recording medium, and the data read by the drive device from the external storage medium is processed by the encryption processing unit. 9 Transfer to 4. By encrypting or decrypting all data to be read or written to or from an external recording medium by hardware at the time of reading or writing, data security can be improved. Various I / O devices connected to the I / O bus 80 are controlled by the second main CPU 60 as necessary.

FIG. 2 is an external view of the information display device 10. The information display device 10 is a horizontally long portable type, and a user operates the information display device 10 with both ends with one hand or both hands.

The information display device 10 is a camera unit 104, LCD 202, analog stick 204, main button 206, start select button 208, left and right speed 210, and a slot for loading and unloading a disc. 2 1 2 and a headphone terminal 2 1 4 are provided. The camera unit 104 faces the same direction as the LCD 202 and shoots a user image.

The camera unit 104 is configured not only to realize the power saving mode characteristic of the present embodiment but also to be used as a normal camera, in which case, one of the main buttons 206 or A dedicated button (not shown) functions as a release button.

The analog stick 204 realizes the function of the joystick in a compact manner. get. The main buttons 206 are △, 〇, X, and mouth buttons, each of which has a specific function assignment. Scan select button 2 08 is a button for starting or selecting a specific process.

FIG. 3 shows a configuration of an estimating unit and a mode control unit characteristic of the present embodiment. In the figure, functional blocks are shown, and do not necessarily correspond to hardware units. Each function is realized by a combination of the first main CPU 30 and other hardware and a control program and other software.

In the figure, the estimating unit 300 includes a user recognizing unit 304 and a gaze determining unit 303. A user image is input to the user recognition unit 304 from the camera unit 104. However, the user recognition unit 304 also receives, as an optional configuration, an “effective area” described later from a ROM (not shown) or the like. The user recognition unit 304 performs an image recognition process on the user image, and determines whether or not the user image has a user face. If there is a face, the area is conveyed to the eye-gaze determination unit 303.

The gaze determination unit 303 determines the gaze direction of the user by examining the face area in more detail, and determines whether the gaze of the user is directed to the LCD 202 based on the result. Any conventional technique can be used for the gaze determination by the cooperation of the user recognition unit 304 and the gaze determination unit 303, and the identification of the method itself is not the purpose of the present invention. An example of gaze determination is as follows.

1. When the user's face recognized by the user recognition unit 304 is near the center of the user image

The gaze determination unit 303 determines the position of both eyes, and when both eyes are substantially at the center of the face, it is determined that the gaze is directed to L g D 202.

2. When the user's face recognized by the user recognition unit 304 is on the right side of the user image

The gaze determination unit 303 determines the position of both eyes. If both eyes are closer to the left side of the face, it is determined that the gaze is facing LCD 202.

3. When the user's face recognized by the user recognition unit 304 is on the left side of the user image.

The gaze determination unit 303 determines the position of both eyes. If both eyes are closer to the right side of the face, it is determined that the gaze is facing LCD 202.

Of course, the line-of-sight determination unit 303 detects the iris of the eye instead of the position of both eyes, The gaze may be detected by a direct method. Known techniques can also be used for iris detection.

As another method, if the user wears a head mount light or a hair band, the orientation of the user's head can be determined based on the position of the head mount and the index provided on them. it can. For example, if a user wears special equipment like a hair band or glasses when playing a special game, a low-illumination infrared LED or other light-emitting element may be installed at a predetermined position in advance. Detecting the head position and direction of the user by providing means such as a fluorescent color indicator or reflector that reflects the light emitted from the information display device 10, and estimating the gaze direction based on them. You may. The above-described effective area may be used in the detection of the user by the user recognition unit 304, particularly in the detection of whether or not the face of the user is present in the user image. The effective area is an image area in which the user may appear in the user image, but it may be determined that the LCD 202 is not actually observed. Area. In particular, if the shooting angle of view of the camera unit 104 is wider than the viewing angle of the LCD 202, the user is not allowed to view the part outside the viewing angle even if the user's face is reflected in the user image. You are not looking at LCD 202. In such a case, an area within the viewing angle of LCD 202 may be set as the effective area.

Further, a threshold value relating to “distance of the user” may be set in the user recognition unit 304. Even if the user is projected in the center of the user image, if the distance is long, it can be considered that the user is not actually looking at the LCD 202. This is especially true when the information display device 10 is portable and the LCD 202 is small. If the user recognition unit 304 recognizes the user's face shown in the user image, but it is determined from the size of the face area that the user distance is, for example, 2 meters, the user recognition unit 304 Does not look at the LCD 202. " However, the user recognition unit 304 does not calculate the distance of the user, but has a threshold value regarding the size of the user's face area, and determines the threshold value and the user's actually projected user value. You only need to compare the size of the face area. 'Either way, the line-of-sight judging unit 303 informs the mode control unit 302 whether or not the user is looking at the LCD 202. User is not looking at LCD 202 During this time, the mode control unit 302 keeps the backlight 102 off. Conversely, while the user is watching the LCD 202, the mode control unit 302 keeps the backlight 102 turned on. The period during which the backlight 102 is turned off is the period of the power saving mode.

An interrupt signal is input to the mode control unit 302 as an optional configuration. This interrupt signal is issued when the user gives an instruction or the like to the information display device 10 such as a keyboard input or a button operation, and the user is looking at the LCD 202. Is to guess. The estimating unit 300 estimates that “the user is not looking at the LCD 202”, and even while the mode control unit 302 is selecting the power saving mode, the interrupt signal is output. Is activated, it is considered that the estimation by the estimating unit 300 may have been incorrect, and as a kind of error processing, the mode control unit 302 sets the information display device 10 to the normal mode. Let me return.

The modes to be realized by the mode control unit 302 need not be the normal mode and the power saving mode, and an intermediate mode can be provided between them, and switching between the normal mode and the power saving mode can be performed. It may be performed continuously by an analog method. In the intermediate mode, the backlight 102 is dimly lit, and in the power saving mode, it is completely turned off. Figures 4 to 7 show examples of mode transition over time. In these figures, “P 0” is a period during which it is estimated that the user is looking at the LCD 202, “P 1” is a period during which it is estimated that the user is not looking at the LCD 202, and time t = 0. Indicates the switching time of those periods, and “N”, “M”, and “L” indicate the normal mode, the intermediate mode, and the power saving mode, respectively. In the case of FIG. 4, the mode control unit 302 immediately shifts to the intermediate mode at t = 0 when the user stops watching the LCD 202, and t = t without the user watching the LCD 202. When it becomes 1, it shifts to the final power saving mode. Of course, when the user begins to look at LCD 202, it returns to normal mode at that time. If the intermediate mode is not provided, it is only necessary to shift to the power saving mode immediately at t = 0.

In the case of FIG. 5, the mode control unit 302 does not immediately shift to the intermediate mode at t = 0 when the user has stopped watching the LCD 202, but waits until t = t2 to shift. Thereafter, if the user reaches t = t3 without looking at the LCD 202, the operation mode is shifted to the final power saving mode. In this case as well, if the user begins to look at LCD 202, the Return to normal mode. The reason for delaying the transition until t = t2 is to assume that the user's line of sight leaves the LCD 202 for a short time.

In the case of FIG. 6, the mode control unit 302 gradually starts shifting to the power saving mode from t = 0 when the user stops watching the LCD 202, and the user switches the LCD 202. If t = t4, the final power saving mode is reached. During this transition, backlight 102 is gradually darkened. According to this method, the mode change is smooth, and the user does not need to be very aware of the mode.

In the case of FIG. 7, the mode control section 302 does not stop the user from seeing the LCD 202 at t = 0, but waits for a while, then gradually shifts to the power saving mode from t = t5. When the user reaches t = t6 without looking at the LCD 202, the final power saving mode is reached. In this case, the effects of FIGS. 5 and 6 are combined.

The present invention has been described based on the embodiments. This embodiment is an exemplification, and it is understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and that such modifications are also within the scope of the present invention. is there.

For example, the power saving mode to be realized by the mode control unit 302 need not be limited to the control of the backlight 102, but it is sufficient if power saving is possible in some sense. Therefore, the power of storage devices, I / O-related devices, and other predetermined electric modules may be turned off, or power may be saved with some reduction in performance. At that time, any known power management can be used.

As another example, the power saving mode by the mode control unit 302 may be a mode for reducing the load of the drawing process on the information display device 10. This method is useful because the display quality can be reduced during the period when the user is not watching LCD 202.

The processing load can be reduced by lowering the spatial detail of the drawing. For this purpose, two modes, that is, modules to be run in the normal mode and the power saving mode, are mounted in the game and other programs to be executed on the information display device 10. The normal mode is a compatible mode, and in hardware that does not need to worry about power saving, different from the image display device 10 according to the embodiment, only the normal mode is executed. Is performed. The default when the image display device 10 is started is set to the normal mode. The first main CPU 30 receives the instruction to shift to the power saving mode via the thread for waiting for the shift instruction. This thread changes the mode flag that the program refers to in various situations from “normal mode” to “power saving mode”. As a result, this program operates in the power saving mode thereafter. In the power saving mode, the spatial detail of the drawing is reduced by the following measures. However, the following measures also depend on the classification method, and may overlap in actual technology.

Lowering the level of detail of LOD (Level Of Details): L0D is a CG method that lowers the level of detail of the model of each object according to the distance from the viewpoint. In the power saving mode, for example, the distance between each object and the viewpoint may be uniformly multiplied by a times (a> l). As a result, the object is recognized as if it were far away and as if it were at a low level of detail. According to this method, it is sufficient to introduce only one parameter a, and the existing program can be utilized as much as possible. If the level of detail of the model decreases, the load of drawing naturally decreases, and the purpose is achieved.

Reduce the number of polygons: Reduce the number of polygons by representing the model in multiple resolutions in advance and expressing it in a simplified model in the power saving mode. If the 3D model is represented by poxels or primitives, the number of them may be reduced. If the model is not multiplexed, a so-called adaptive subdivision method is used, such as combining small polygons and poxels into adjacent polygons ^ poxels to generate a simplified model, and then using the simplified model. You may. The same is true for Poxels.

Simplify surface representation: If the surface of an object is represented by a surface such as a NURBS surface or other free-form surface, reduce the number of control points on the surface to reduce the number of parameters or divide the surface Simplify the presentation by reducing the number of patches.

Reduce the number of effects: For example, if the scene of the explosion is represented by many particles, reduce the number of particles. The same applies to metaballs and the like. Shallow view volume: Reduce the number of objects that enter the drawing target space by placing the far plane of the view volume relatively close to the viewpoint.

Conceal objects: for example, when the fog appears in the space to be Generate a state that is no longer there. As a result, the number of objects to be drawn is reduced, or simple drawing such as expressing with one color becomes possible. '

Lower the degree of the space: 'Draw the three-dimensional space that should be drawn as a two-dimensional space. For this purpose, fix the viewpoint and other camera parameters.

Lighten the shading: Make the shading solid, simple, or stop shading itself.

Reduce screen size: Reduce the size of the image displayed on the screen by the display control circuit of the drawing unit 26. This reduces the number of pixels to be processed for display, thus reducing the processing load. The screen size may remain the same and the display resolution may be reduced. In this case, the same effect is obtained. In this method, the program being executed may be an existing program that does not support the power saving mode. This is because the configuration of the image display device 10, such as one of the first main CPU 30, the second main CPU 60, and the MPEG decoder 56, is designed to reduce the display resolution in the power saving mode. That's all you need to do.

The examples in which the first main CPU 30 and the drawing processing unit 26 cooperate have been described above.In addition, according to the cooperation of the second main CPU 60 and the MPEG decoder 56, the following measures are taken. Is also possible.

Force spatial frequency components: J PEG (Joint Photographic

When encoding and decoding an image by decomposing it into spatial frequency components, such as Expert Group) and MPEG for moving images, the subjective quality of the image is mainly determined by the low frequency components. Therefore, decoding power can be reduced by stopping the decoding process at a certain threshold frequency or by truncating and displaying the frequency components decoded in a relatively short time. The program does not need to support the power saving mode, and the MPEG decoder 56 only needs to recognize and execute the power saving mode. The above is the method of reducing the spatial detail level of drawing.

Next, there is a method of reducing the temporal detail of the drawing. First, the following measures are possible by the cooperation of the first main CPU 30 and the drawing processing unit 26.

Decrease the frame rate: For example, by reducing the output of 30 frames / sec to 15 frames / sec, a significant power reduction can be realized in the display system. In this case, Since the processing in the MPEG decoder 56 is also reduced, the power reduction effect is high. In this measure, the running program does not need to support the power saving mode, and the MPEG decoder 56, the drawing unit 26, and the like need only recognize and execute the power saving mode.

Simplify the movement of the object: Only the movement of the entire object on the screen can be expressed, and the movement of each part of the object can be fixed.

The following measures are possible by the cooperation of the second main CPUs 60 and 56. Display only intra-coded pictures: In the case of MPEG, I-pictures do not need to refer to other pictures, so drawing only I-pictures in a frame-by-frame manner can reduce the drawing process. In this case, the running program does not need to support the power saving mode, and the MPEG decoder 56, the drawing unit 26, and the like need only recognize and execute the power saving mode. Industrial applicability

The present invention can be applied to an image display device having a display device.

Claims

The scope of the claims

1. Display device,

A camera unit for photographing a direction in which a user is assumed to be present;

An estimating unit that estimates whether or not the user is looking at the display device based on an image captured by the power camera unit;

When the estimation unit estimates that the user does not look at the display device, a mode control unit that performs a transition to a power saving mode;

An image display device comprising:

2. The image display device according to claim 1, wherein the power saving mode is a mode in which a power supply of a predetermined electric module is turned off.

3. The image display device according to claim 2, wherein the power saving mode is a mode for turning off a backlight of the display device.

4. The apparatus according to claim 1, wherein the mode control unit performs a step-by-step transition to the power saving mode in a stepwise manner according to a time when the user is estimated not to be looking at the display device. apparatus.

5. The image display device according to claim 1, wherein the power saving mode is a mode for reducing a load of a drawing process.