Commit | Line | Data |
---|---|---|
a09e091a JB |
1 | <?xml version="1.0" encoding="ISO-8859-1"?> |
2 | <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN" | |
3 | "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [ | |
4 | <!ENTITY % defs SYSTEM "/xserver/doc/xml/xserver.ent"> %defs; | |
5 | ]> | |
6 | <article> | |
7 | ||
8 | <articleinfo> | |
9 | <!-- Title information --> | |
10 | <title>Scaled Window Support in DMX</title> | |
11 | <authorgroup> | |
12 | <author><firstname>Kevin E.</firstname><surname>Martin</surname></author> | |
13 | <author><firstname>Rickard E.</firstname><surname>Faith</surname></author> | |
14 | </authorgroup> | |
15 | <pubdate>15 October 2003 (created 19 September 2003)</pubdate> | |
16 | <releaseinfo>X Server Version &xserver.version;</releaseinfo> | |
17 | <abstract> | |
18 | <para> | |
19 | This document investigates the possibility of adding scaled window | |
20 | support to the DMX X server, thereby allowing a window or some | |
21 | selected part of the logical DMX area to be displayed using a | |
22 | scaling factor. For example, this might allow the contents of a | |
23 | window to be magnified for easier viewing. In particular, scaling | |
24 | for the VNC client is explored. <emphasis remap="it">Copyright 2003 | |
25 | by Red Hat, Inc., Raleigh, North Carolina</emphasis> | |
26 | </para> | |
27 | </abstract> | |
28 | </articleinfo> | |
29 | ||
30 | <!-- Begin the document --> | |
31 | <sect1><title>Introduction</title> | |
32 | <sect2><title>DMX</title> | |
33 | <para> | |
34 | The DMX X server (Xdmx) is a proxy server that is designed | |
35 | to allow X servers on multiple machines to be combined into | |
36 | a single multi-headed X server. Combined with Xinerama, | |
37 | these heads can appear as a single very high-resolution | |
38 | screen. Typical applications include the creation of a | |
39 | video wall with 16 1280x1024 displays arranged in a | |
40 | rectangle, for a total resolution of of 5120x4096. | |
41 | </para> | |
42 | </sect2> | |
43 | <sect2><title>Problem Statement</title> | |
44 | <para> | |
45 | Applications displayed on a physically large video wall that | |
46 | provides high pixel-resolution may be difficult to see, | |
47 | especially if the application is designed for use on a | |
48 | typical desktop computer with a relatively small display | |
49 | located close to the human operator. The goal of this paper | |
50 | is to describe and discuss solutions to this problem. | |
51 | </para> | |
52 | <para> | |
53 | The original driving problem for this work is to provide | |
54 | scaling for the <command>vncviewer</command> application when | |
55 | displayed using DMX (VNC scaling is currently available only | |
56 | with the Windows client, and there is no plan to extend that | |
57 | capability to other clients). While this specific problem | |
58 | will be addressed in this paper, the general solution space | |
59 | will also be explored, since this may lead to a good | |
60 | solution not only for <command>vncviewer</command> but also for | |
61 | other applications. | |
62 | </para> | |
63 | </sect2> | |
64 | <sect2><title>Task</title> | |
65 | <para> | |
66 | For reference, here is the original description of the task | |
67 | this paper addresses: | |
68 | <itemizedlist> | |
69 | <listitem><para>Scaled window support (for VNC) | |
70 | <itemizedlist> | |
71 | <listitem><para> | |
72 | Investigate possibility of implementing a "scaled | |
73 | window" extension: | |
74 | <itemizedlist> | |
75 | <listitem><para> | |
76 | Add XCreateScaledWindow call that could be used | |
77 | in place of XCreateWindow | |
78 | </para></listitem> | |
79 | <listitem><para> | |
80 | All primitives drawn to scaled window would be | |
81 | scaled by appropriate (integral?) scaling factor | |
82 | </para></listitem> | |
83 | </itemizedlist> | |
84 | </para></listitem> | |
85 | <listitem><para> | |
86 | Alternate approach: special case VNC support | |
87 | </para></listitem> | |
88 | </itemizedlist> | |
89 | </para></listitem> | |
90 | </itemizedlist> | |
91 | </para> | |
92 | </sect2> | |
93 | </sect1> | |
94 | ||
95 | <sect1><title>Previous Work</title> | |
96 | <para> | |
97 | This section reviews relevant previous work. | |
98 | </para> | |
99 | <sect2><title>VNC</title> | |
100 | <sect3><title>Scaling under VNC</title> | |
101 | <para> | |
102 | When using the <command>vncviewer</command> program for Windows, it | |
103 | is possible to specify a scaling factor (as numerator and | |
104 | denominator). When scaling is in effect, the viewer | |
105 | software uses StretchBlt (instead of BitBlt) to display | |
106 | the pixels for the user. When this call is made, the | |
107 | viewer already has received all of the pixel information | |
108 | (at full unscaled resolution). | |
109 | </para> | |
110 | <para> | |
111 | The scaling in VNC is primitive. It does not conserve | |
112 | bandwidth, it does not treat textual information | |
113 | differently (i.e., by using a suitably scaled font), and | |
114 | it does not provide any anti-aliasing other than that | |
115 | provided by the underlying (Windows-only) system library. | |
116 | </para> | |
117 | </sect3> | |
118 | </sect2> | |
119 | <sect2><title>The X Video Extension</title> | |
120 | <para> | |
121 | The X Video Extension is a widely-available extension to the | |
122 | X11 protocol that provides support for streaming video. | |
123 | Integral to this support is the ability to arbitrarily scale | |
124 | the output. In version 2.2 of the X Video specification, | |
125 | support for scaled still images was provided, using both | |
126 | shared memory and traditional transport. The API for this | |
127 | support uses calls that are quite similar to XCreateWindow, | |
128 | XPutImage, and XShmPutImage. Currently, most of the drivers | |
129 | implemented in XFree86 only support data in various YUV | |
130 | formats. However, several modern video adaptors support RGB | |
131 | as well. | |
132 | </para> | |
133 | <para> | |
134 | Note, though, that the target output for this scaling is an | |
135 | overlay plane -- so X Video provides functionality that is | |
136 | fundamentally different from that provided by the Windows | |
137 | StrechBlt call. | |
138 | </para> | |
139 | </sect2> | |
140 | </sect1> | |
141 | ||
142 | <sect1><title>Possible Solutions</title> | |
143 | <para> | |
144 | This section briefly discusses possible solutions, including | |
145 | major advantages and disadvantages from both the | |
146 | implementation and the end-user programmer standpoint. | |
147 | </para> | |
148 | <sect2><title>VNC-like Scaling</title> | |
149 | <sect3><title>Software Scaling</title> | |
150 | <para> | |
151 | The <command>vncviewer</command> application could be modified to | |
152 | provide software scaling. This is not a general solution, | |
153 | but it does solve one of the goals of this work. | |
154 | </para> | |
155 | <para> | |
156 | A prototype of this solution was implemented and a patch | |
157 | against <filename>vnc-3.3.7-unixsrc</filename> is available in the | |
158 | <filename>dmx/external</filename> directory. Because of limited time | |
159 | available for this work, all of the edge cases were not | |
160 | considered and the solution works well mainly for integer | |
161 | scaling. | |
162 | </para> | |
163 | <para> | |
164 | Currently, <command>vncviewer</command> writes to the X display | |
165 | with XPutImage, XCopyArea, and XFillRectangle. All | |
166 | instances of these calls have to be aware of scaling | |
167 | and must round correctly. In the prototype solution, | |
168 | rounding is incorrect and can cause artifacts. | |
169 | </para> | |
170 | <para> | |
171 | A better solution would be to cache all updates to the | |
172 | desktop image in <command>vncviewer</command> and only send the | |
173 | damaged area to the X display with XPutImage. This would | |
174 | allow the damaged area to be computed so that rounding | |
175 | errors do not create artifacts. This method is probably | |
176 | similar to what is used in the Window client. (The whole | |
177 | VNC suite is being re-written in C++ and the forthcoming | |
178 | version 4 has not been evaluated.) | |
179 | </para> | |
180 | </sect3> | |
181 | <sect3><title>Scaling with the X Video Extension</title> | |
182 | <para> | |
183 | The scaling in the Windows <command>vncviewer</command> application | |
184 | makes use of a scaled blit that is supplied by the | |
185 | underlying system library. Several video cards currently | |
186 | provide support for a scaled blit, and some X servers | |
187 | (including XFree86) expose this capability to applications | |
188 | via the XvPutImage interface of the X Video Extension. | |
189 | The capability exposed by XvPutImage results in the scaled | |
190 | image being drawn to an overlay plane. Most video cards | |
191 | also provide support for a scaled blit into the normal | |
192 | output planes, but this is not exposed via XvPutImage. | |
193 | </para> | |
194 | <para> | |
195 | The <command>vncviewer</command> program could be modified to use | |
196 | the X Video Extension to provide scaling under X11 that is | |
197 | similar to the scaling currently provided under Windows. | |
198 | Unfortunately, Xdmx does not currently export the X Video | |
199 | Extension, so this would not provide an immediate solution | |
200 | usable with DMX. | |
201 | </para> | |
202 | <para> | |
203 | A very early-stage proof-of-concept prototype was | |
204 | implemented and a preliminary patch against | |
205 | <filename>vnc-3.3.7-unixsrc</filename> is available in the | |
206 | <filename>dmx/external</filename> directory. This prototype was | |
207 | implemented to better understand the problems that must be | |
208 | solved to make this solution viable: | |
209 | <itemizedlist> | |
210 | <listitem><para> | |
211 | As noted under the software scaling section above, | |
212 | <command>vncviewer</command> writes to the X display with | |
213 | several different calls. These calls write to the | |
214 | normal output planes and are compatible with | |
215 | XvPutImage, which writes to an overlay plane. To | |
216 | eliminate artifacts caused by this problem, | |
217 | <command>vncviewer</command> should be modified so that a cached | |
218 | copy of the desktop is available, either as a | |
219 | client-side image or a server-side off-screen pixmap, | |
220 | so that XvPutImage would be the only method for | |
221 | writing to the X display. | |
222 | </para></listitem> | |
223 | <listitem> | |
224 | <para> | |
225 | Although several modern graphics adaptors support | |
226 | hardware scaling using an RGB format (e.g., ATI | |
227 | Radeon, nVidia, etc.), XFree86 drivers typically | |
228 | only implement YUV formats. YUV generally compress | |
229 | the pixel information in some way. For example, two | |
230 | commonly implemented formats, YUY2 and UYVY provide | |
231 | intensity information for every RGB pixel, but only | |
232 | provide chroma and luminance information for pairs | |
233 | of horizontal pixels. Since VNC uses | |
234 | pixel-resolution for communicating updates on the | |
235 | wire, additional artifacts are introduced (because | |
236 | there may not be enough information from the wire to | |
237 | update a pair of pixels). | |
238 | </para> | |
239 | <para> | |
240 | Further, the well-known problem with YUV encoding | |
241 | is even more evident when the image is a desktop | |
242 | instead of a movie. For example, consider a | |
243 | 1-pixel-wide vertical window border. If the border | |
244 | changes in color but not intensity (e.g., because a | |
245 | window manager uses color to indicate focus), there | |
246 | may or may not be a change in the YUY2 image, | |
247 | depending on the algorithm used for RGB to YUV | |
248 | conversion and on how the border pixel is ordered in | |
249 | the pair of pixels used by the algorithm. | |
250 | </para> | |
251 | <para> | |
252 | Many of these artifacts could be eliminated if | |
253 | <command>vncviewer</command> cached a complete RGB image of | |
254 | the desktop, and only did the conversion to YUV for | |
255 | properly aligned areas of damage. The remaining artifacts | |
256 | could be eliminated if an RGB format was used with X | |
257 | Video (which may require the extension of existing | |
258 | XFree86 drivers to support RGB). | |
259 | </para> | |
260 | </listitem> | |
261 | <listitem><para> | |
262 | Most modern video cards support exactly one overlay | |
263 | plane that is suitable for use with X Video. | |
264 | Therefore, only one application can use X Video at any | |
265 | given time. This is a severe limitation in a desktop | |
266 | environment. | |
267 | </para></listitem> | |
268 | </itemizedlist> | |
269 | </para> | |
270 | <sect4><title>Implementing the X Video Extension for DMX</title> | |
271 | <para> | |
272 | The user-level API for X Video is fairly simple, but the | |
273 | underlying support required for the full specification | |
274 | is large. However, since the API provides a method to | |
275 | query supported capabilities, a usable subset of X | |
276 | Video can be implemented that would support XvPutImage | |
277 | and little else. This would require support for the | |
278 | following: | |
279 | <itemizedlist> | |
280 | <listitem><para> | |
281 | X Video Extension API calls, including the | |
282 | following: | |
283 | <itemizedlist> | |
284 | <listitem><para>XvQueryExtension</para></listitem> | |
285 | <listitem><para>XvQueryAdaptors</para></listitem> | |
286 | <listitem><para>XvQueryPortAttributes</para></listitem> | |
287 | <listitem><para>XvFreeAdaptorInfo</para></listitem> | |
288 | <listitem><para>XvListImageFormats</para></listitem> | |
289 | <listitem><para>XvGrabPort</para></listitem> | |
290 | <listitem><para>XvCreateImage</para></listitem> | |
291 | <listitem><para>XvPutImage</para></listitem> | |
292 | <listitem><para>XvShmCreateImage</para></listitem> | |
293 | <listitem><para>XvShmPutImage</para></listitem> | |
294 | </itemizedlist> | |
295 | </para></listitem> | |
296 | <listitem><para> | |
297 | Support for querying back-end X Video Extension | |
298 | capabilities. | |
299 | </para></listitem> | |
300 | <listitem><para> | |
301 | Support for sending the image to the back-ends. | |
302 | Because X Video requires sending full images, there | |
303 | may be a trade-off between bandwidth limitations and | |
304 | additional complexity to divide the image up such | |
305 | that is scales properly. | |
306 | </para></listitem> | |
307 | <listitem><para> | |
308 | Possible support for a software fall-back. For | |
309 | example, if all of the back-ends do not support the X | |
310 | Video Extension, software scaling can be implemented | |
311 | such that the image is sent to the back-end with | |
312 | XPutImage. This pathway would have poor | |
313 | performance. | |
314 | </para></listitem> | |
315 | </itemizedlist> | |
316 | </para> | |
317 | </sect4> | |
318 | <sect4><title>Supporting RGB formats for the X Video Extension</title> | |
319 | <para> | |
320 | Assuming an XFree86 driver already supports the X Video | |
321 | Extension, and assuming the target hardware supports an | |
322 | RGB format, then adding support for that format is | |
323 | relatively simple and straightforward. | |
324 | </para> | |
325 | </sect4> | |
326 | </sect3> | |
327 | <sect3><title>Scaling with an XPutImageScaled Extension</title> | |
328 | <para> | |
329 | Instead of (or in addition to) implementing the X Video | |
330 | Extension in DMX, one obvious solution would be to | |
331 | implement a new extension that provides access to | |
332 | hardware-assisted scaled blits, similar to the StretchBlt | |
333 | call available under Windows. This call would scale RGB | |
334 | images and would not use the overlay plane (unlike the X | |
335 | Video Extension). | |
336 | </para> | |
337 | <para> | |
338 | This approach has many of the same advantages and | |
339 | disadvantages as the XCopyAreaScaled Extension, discussed | |
340 | in the next section. Discussion of XPutImageScaled is | |
341 | deferred in favor of XCopyAreaScaled for the following | |
342 | reasons: | |
343 | <itemizedlist> | |
344 | <listitem><para> | |
345 | XPutImageScaled can be emulated with XCopyAreaScaled | |
346 | by first using XPutImage to copy the image to an | |
347 | off-screen pixmap, and then calling XCopyAreaScaled | |
348 | between that off-screen pixmap and the target | |
349 | drawable. | |
350 | </para></listitem> | |
351 | <listitem><para> | |
352 | Since XCopyAreaScaled would copy between two areas of | |
353 | on-screen or off-screen memory, it has additional uses | |
354 | and can be viewed as efficiently providing a superset | |
355 | of XPutImageScaled functionality. | |
356 | </para></listitem> | |
357 | </itemizedlist> | |
358 | </para> | |
359 | </sect3> | |
360 | <sect3><title>Scaling with an XCopyAreaScaled Extension</title> | |
361 | <para> | |
362 | As noted in the previous section, because XCopyAreaScaled | |
363 | provides a superset of the functionality provided by | |
364 | XPutImageScaled, we will consider this extension instead. | |
365 | </para> | |
366 | <para> | |
367 | First, XCopyAreaScaled would provide for RGB scaling | |
368 | between pixmaps (i.e., on-screen or off-screen areas of | |
369 | memory that reside on the video card). Unlike the X Video | |
370 | Extension, which writes into an overlay plane, | |
371 | XCopyAreaScaled would write into the non-overlay areas of | |
372 | the screen. Key points to consider are as follows: | |
373 | <itemizedlist> | |
374 | <listitem><para> | |
375 | Because different planes are involved, the two scaling | |
376 | operations are usually implemented in hardware | |
377 | differently, so an XCopyAreaScaled extension could be | |
378 | added in a manner that would neither conflict with nor | |
379 | interact with the X Video extension in any way. | |
380 | </para></listitem> | |
381 | <listitem><para> | |
382 | The XCopyAreaScaled extension provides new | |
383 | functionality that the X Video Extension does not | |
384 | provide. Based on anecdotal feedback, we believe that | |
385 | many people outside the DMX and VNC communities would | |
386 | be excited about this extension. | |
387 | </para></listitem> | |
388 | <listitem><para> | |
389 | The main drawback to this extension is that it is new | |
390 | and needs to be implemented at the driver level in | |
391 | XFree86 for each video card to be supported. At the | |
392 | present time, it is more likely that the X Video | |
393 | Extension will be implemented for a particular piece | |
394 | hardware because the X Video extension has multimedia | |
395 | uses. However, over time, we would expect the | |
396 | XCopyAreaScaled extension to be implemented along with | |
397 | the X Video extension, especially if it becomes | |
398 | popular. | |
399 | </para></listitem> | |
400 | <listitem><para> | |
401 | Another drawback is that not all modern cards provide | |
402 | support for a simple scaled blit operation. However, | |
403 | these cards usually do provide a 3D pipeline which | |
404 | could be used to provide this functionality in a | |
405 | manner that is transparent to the client application | |
406 | that is using the XCopyAreaScaled extension. However, | |
407 | this implementation pathway would make this extension | |
408 | somewhat more difficult to implement on certain cards. | |
409 | </para></listitem> | |
410 | </itemizedlist> | |
411 | </para> | |
412 | </sect3> | |
413 | <sect3><title>Scaling with OpenGL</title> | |
414 | <para> | |
415 | Another general solution to the scaling problem is to use | |
416 | the texture scaling found in all 3D hardware. This | |
417 | ability is already exposed through OpenGL and can be | |
418 | exploited by clients without X server modification (i.e., | |
419 | other than the ability to support OpenGL). An application | |
420 | using OpenGL would transmit the non-scaled image to the X | |
421 | server as a texture, and would then display a single | |
422 | non-transformed rect using that texture. This also works | |
423 | around the single overlay problem with the X Video | |
424 | Extension as well as the need to implement additional | |
425 | scaled primitive extensions. | |
426 | </para> | |
427 | <para> | |
428 | The downside is that most OpenGL implementations require | |
429 | power of 2 texture sizes and this can be very wasteful of | |
430 | memory if, for example, the application needs to scale a | |
431 | 1025x1025 image, which would require a 2048x2048 texture | |
432 | area (even a 640x480 image would require a 1024x512 | |
433 | texture). Another downside is that some OpenGL | |
434 | implementations have a limited about of texture memory and | |
435 | cannot handle textures that are very large. For example, | |
436 | they might limit the texture size to 1024x1024. | |
437 | </para> | |
438 | </sect3> | |
439 | </sect2> | |
440 | <sect2><title>Application-transparent Scaling for DMX | |
441 | </title><sect3><title>Back-end Scaling Without Disconnect/Reconnect</title> | |
442 | <para> | |
443 | VNC does scaling on the client side (in the | |
444 | <command>vncviewer</command> application). Implementing a similar | |
445 | solution for DMX would require support in the back-end X | |
446 | servers and, therefore, is not a general solution. | |
447 | </para> | |
448 | <para> | |
449 | XFree86 already implements some support for "scaling" that | |
450 | could be used with DMX: if, in the XF86Config file, | |
451 | multiple Modes are listed in the Display Subsection of the | |
452 | Screen Section, then pressing Ctrl-Alt-Plus and | |
453 | Ctrl-Alt-Minus can be used to iterate through the listed | |
454 | modes. The display dimensions will change to the | |
455 | dimensions in the Modes line, but the logical dimensions | |
456 | of the X server (i.e., the dimensions that Xdmx knows | |
457 | about) will not change. | |
458 | </para> | |
459 | <para> | |
460 | Further, the dimensions of the XFree86 display are under | |
461 | software control (via the XFree86-VidModeExtension), so | |
462 | the Xdmx server could change the screen dimensions on a | |
463 | per-display basis, thereby scaling the information on part | |
464 | of that display. | |
465 | </para> | |
466 | <para> | |
467 | However, this scaling appears to have limited use. For | |
468 | example, assume a 4 by 4 display wall consisting of 16 | |
469 | 1280x1024 displays. If all of the back-end servers were | |
470 | simultaneously configured to display 640x480, the left | |
471 | hand corner of each display would be magnified, but the | |
472 | composite result would be unreadable. Magnifying one | |
473 | display at a time could be usable, but could have limited | |
474 | utility, since the result would still be no larger than a | |
475 | single display. | |
476 | </para> | |
477 | </sect3> | |
478 | <sect3><title>Back-end Scaling With Disconnect/Reconnect</title> | |
479 | <para> | |
480 | Disconnect and reconnect features are not currently | |
481 | supported in DMX, but are scheduled to be implemented in | |
482 | the future. These features, combined with the | |
483 | XFree86-VidModeExtension Extension, would allow an | |
484 | application to do the following: | |
485 | <itemizedlist> | |
486 | <listitem><para> | |
487 | Disconnect a specific back-end server (via the DMX | |
488 | Extension), | |
489 | </para></listitem> | |
490 | <listitem><para> | |
491 | reconfigure the XFree86 back-end server resolution, | |
492 | and | |
493 | </para></listitem> | |
494 | <listitem><para> | |
495 | reconnect the back-end server to DMX -- at a new | |
496 | origin with the new screen resolution. | |
497 | </para></listitem> | |
498 | </itemizedlist> | |
499 | </para> | |
500 | <para> | |
501 | For example, consider a display wall consisting of 16 | |
502 | 1280x1024 displays with a total resolution of 5120x4096. | |
503 | All of the screens could be disconnected, repositioned, | |
504 | and reconnected each at a resolution of 640x480. The | |
505 | total resolution of the display wall would be 2560x1920, | |
506 | allowing a view of a selected area approximately | |
507 | one-fourth of the size of the DMX display. This change | |
508 | would be completely application independent (except, | |
509 | perhaps, for a DMX-aware window manager). When work at | |
510 | the increased resolution was completed, the back-end | |
511 | servers could be disconnected, reconfigured, and | |
512 | reconnected for the original 5120x4096 view. | |
513 | </para> | |
514 | <para> | |
515 | Support for this type of scaling can be implemented in a | |
516 | DMX-aware X11 client assuming the DMX server support | |
517 | arbitrary disconnect and reconnect semantics. Because | |
518 | this application cannot be written before | |
519 | disconnect/reconnect is implemented, this solution will | |
520 | not be discussed further in this paper. | |
521 | </para> | |
522 | </sect3> | |
523 | <sect3><title>Server-side Scaling</title> | |
524 | <para> | |
525 | In earlier versions of DMX, a frame buffer was maintained | |
526 | on the server side, and XPutImage was used to move the | |
527 | information from the server to the client (similar to some | |
528 | early VNC implementations). The use of a server-side | |
529 | frame buffer would allow the server to do scaling, but is | |
530 | not a recommended solution because of overall performance | |
531 | issues and server-side memory issues (i.e., the frame | |
532 | buffer would be very large for large display walls). | |
533 | </para> | |
534 | <para> | |
535 | Exploration of this path is not recommended. | |
536 | </para> | |
537 | </sect3> | |
538 | </sect2> | |
539 | <sect2><title>XCreateScaledWindow API</title> | |
540 | <para> | |
541 | The implementation of X Video Extension in DMX, and the use | |
542 | of XvPutImage by applications requiring scaling requires | |
543 | significant changes in DMX Further, XvPutImage is, | |
544 | essentially a scaled blit, and it is only useful for | |
545 | applications which are already using (or can be modified to | |
546 | use) XPutImage. Therefore, a more general API will be | |
547 | discussed as another possibility. | |
548 | </para> | |
549 | <para> | |
550 | X applications typically create windows with the | |
551 | XCreateWindow call. A new extension could provide an | |
552 | XCreateScaledWindow call that could be used in place of the | |
553 | XCreateWindow call and be otherwise transparent to the | |
554 | application. This would allow applications, even those that | |
555 | do not depend on XPutImage, to take advantage of window | |
556 | scaling. In this section we describe how the call would | |
557 | work, what transparency it provides, and how to solve the | |
558 | potential problems that transparency creates. | |
559 | </para> | |
560 | <sect3><title>XCreateWindow</title> | |
561 | <para> | |
562 | The XCreateWindow call takes width and height as | |
563 | parameters. An XCreateScaledWindow call could take all | |
564 | the same parameters, with the addition of a scaling factor. | |
565 | </para> | |
566 | </sect3> | |
567 | <sect3><title>XSetWindowAttributes</title> | |
568 | <para> | |
569 | An X11 window has several attributes that would have to be | |
570 | scaled: | |
571 | <itemizedlist> | |
572 | <listitem><para>Background and border pixmaps</para></listitem> | |
573 | <listitem><para>Border width</para></listitem> | |
574 | <listitem><para>Cursor</para></listitem> | |
575 | </itemizedlist> | |
576 | </para> | |
577 | </sect3> | |
578 | <sect3><title>XGetWindowAttributes, XGetGeometry</title> | |
579 | <para> | |
580 | For transparency, calls that query the window attributes | |
581 | should return unscaled information. This suggests that | |
582 | all unscaled pixmaps and window attributes should be | |
583 | cached. | |
584 | </para> | |
585 | <para> | |
586 | Unfortunately, a window manager requires the scaled | |
587 | geometry to properly decorate the window. The X server | |
588 | can probably determine which client is acting as the | |
589 | window manager (e.g., because that client will select | |
590 | events that are used exclusively by the window manager). | |
591 | However, other Scaled Window Extension aware clients may | |
592 | also need to determine the scaled geometry. Therefore, at | |
593 | least two additional extension calls should be | |
594 | implemented: XGetScaledWindowAttributes and | |
595 | XGetScaledGeometry. | |
596 | </para> | |
597 | </sect3> | |
598 | <sect3><title>Popup and Child window positions</title> | |
599 | <para> | |
600 | Some applications may position popup and child windows | |
601 | based on an unscaled notion of the main window geometry. | |
602 | In this case, additional modifications to the client would | |
603 | be required. | |
604 | </para> | |
605 | </sect3> | |
606 | <sect3><title>Events</title> | |
607 | <para> | |
608 | Most events (e.g., for mouse motion) return information | |
609 | about the coordinates at which the even occurred. These | |
610 | coordinates would have to be modified so that unscaled | |
611 | values were presented to the client. | |
612 | </para> | |
613 | </sect3> | |
614 | <sect3><title>Implementation</title> | |
615 | <para> | |
616 | There are many implementation issues, some of which are | |
617 | similar to the issues involved in implementing the X Video | |
618 | Extension for DMX. The window contents must be scaled, | |
619 | either by performing all operations to a frame buffer and | |
620 | then writing the image to the display (perhaps using | |
621 | hardware scaling support), or by modifying all of the | |
622 | various drawing operations to perform scaling. Because of | |
623 | the complexity involved, the frame buffer option is | |
624 | recommended. | |
625 | </para> | |
626 | </sect3> | |
627 | </sect2> | |
628 | </sect1> | |
629 | ||
630 | <sect1><title>Conclusion and Recommendations | |
631 | </title><para> | |
632 | We recommend a three phase implementation strategy, based on | |
633 | how an application could be written to take advantage of | |
634 | scaling: | |
635 | <orderedlist> | |
636 | <listitem> | |
637 | <para> | |
638 | The XCopyAreaScaled extension should be implemented, since | |
639 | this is the ideal solution for applications like VNC, and | |
640 | since making use of this extension will require minimal | |
641 | changes to applications that already use XPutImage or | |
642 | XCopyArea. | |
643 | </para> | |
644 | <para> | |
645 | The initial implementation work would include the design | |
646 | of the X protocol extension, writing this up in the | |
647 | usual format for extension documentation, implementation | |
648 | of the protocol transport pieces in XFree86, | |
649 | implementation of a software fall-back in XFree86 and | |
650 | DMX, one example hardware implementation for XFree86, | |
651 | and implementation of support for this extension in DMX. | |
652 | </para> | |
653 | <para> | |
654 | We suggest implementing the extension first on the ATI | |
655 | Radeon cards. However, since these cards do not provide | |
656 | a 2D scaled blit primitive, the implementation would | |
657 | have to make use of the 3D texture engine to emulate a | |
658 | scaled blit. This is recommended, since other modern | |
659 | graphics cards also do not provide a simple 2D scaled | |
660 | blit operation and an example of the more difficult | |
661 | implementation pathway would be helpful to others. | |
662 | </para> | |
663 | </listitem> | |
664 | <listitem> | |
665 | <para> | |
666 | Until XCopyAreaScaled is widely supported, applications | |
667 | that require scaling will have to fall back to another | |
668 | scaling method. We suggest OpenGL as the first fall-back | |
669 | method because it is widely available and supported by | |
670 | DMX. | |
671 | </para> | |
672 | <para> | |
673 | A project centered around OpenGL-based scaling would | |
674 | implement this scaling in VNC as an example. This work | |
675 | would include re-writing the <command>vncviewer</command> | |
676 | rendering engine to cache a master copy of the desktop | |
677 | image for all operations. | |
678 | </para> | |
679 | </listitem> | |
680 | <listitem> | |
681 | <para> | |
682 | Since OpenGL is not implemented everywhere, and may not | |
683 | provide hardware-assisted performance in every | |
684 | implementation, an application that requires scaling | |
685 | should also fall back to using the X Video Extension. | |
686 | </para> | |
687 | <para> | |
688 | This project would add support for the X Video Extension | |
689 | to DMX and would add support to VNC to take advantage of | |
690 | this extension without introducing artifacts. This | |
691 | would require modifying the <command>vncviewer</command> rendering | |
692 | engine to cache a master copy of the desktop image for | |
693 | all operations. This project should also add support | |
694 | for the RGB format to at least one XFree86 driver (e.g., | |
695 | ATI Radeon). | |
696 | </para> | |
697 | <para> | |
698 | The X Video Extension is one of the few popular | |
699 | extensions that DMX does not support. We recommend | |
700 | implementing the X Video Extension even if scaling is | |
701 | the specific goal of that work. | |
702 | </para> | |
703 | </listitem> | |
704 | </orderedlist> | |
705 | </para> | |
706 | <para> | |
707 | We do <emphasis>not</emphasis> recommend implementation of the | |
708 | XCreateScaledWindow extension because of the complexity | |
709 | involved. We do <emphasis>not</emphasis> recommend implementation of the | |
710 | XPutImageScaled extension because it requires the same amount | |
711 | of work as the XCopyAreaScaled extension, but provides less | |
712 | functionality. Further, server-side scaling with a large | |
713 | frame buffer is <emphasis>not</emphasis> recommended because of the | |
714 | performance implications. | |
715 | </para> | |
716 | <para> | |
717 | The back-end scaling, especially with disconnect/reconnect | |
718 | support should be explored in the future after | |
719 | disconnect/reconnect is implemented, but not at the present | |
720 | time. | |
721 | </para> | |
722 | </sect1> | |
723 | ||
724 | </article> | |
725 | <!-- Local Variables: --> | |
726 | <!-- fill-column: 72 --> | |
727 | <!-- End: --> |