Skip Navigation

Friday August 2nd, 2013

OIT > Computing > DCS - Unix Support > Planned Changes

DCS Unix Support Services

Service & Cost Structure and other Changes
September, 1999

Summary: This document describes the current status of OIT's Distributed Computing Support (DCS) services, and summarizes planned changes. The changes are in response to the evolving needs of the campus, and are designed to improve the overall quality of the service. Among other things, a new price structure is planned where fees will be more closely related to the actual staff time required for each type of service.

Table of Contents

Introduction

DCS has served the campus for over 10 years, providing professional Unix services within limited campus support resources. Thousands of Unix users would not have had support without DCS. DCS has transparently recruited, trained, and maintained a capable support staff despite significant turbulence in the computer professional marketplace. Currently, DCS’s major clients are Physical Sciences and the School of Engineering. DCS also supports workgroups in the Administration, Biological Sciences, the College of Medicine, Social Sciences, Humanities, and elsewhere on campus. A total of over 260 computing servers and systems used by over 4,500 end-users, are under DCS care. (This does not include the systems supported by DCS-assisted local staff).

As support demands continue to grow and evolve, it is important to review DCS support structure and issues to determine adjustments required. OIT has studied the situation and discussed options with representative DCS clients . The enclosed plans are the result of this study.

Top of Page

Issues

DCS supports 260 computing systems and servers utilized by over 4,500 users, day-in and day-out. The majority of the service works well; changes made to improve the service should be done carefully to preserve existing benefits. The issues needing to be addressed are discussed below.

Client Perceptions and DCS Responsiveness

Although most DCS clients are satisfied with the service they receive, there are some that have concerns about responsiveness and other issues. There clearly is opportunity to enhance the service.

Faculty clients are not typically aware of the amount of time spent on "infrastructure;" these behind-the-scenes activities distract staff from being more communicative and responsive to direct faculty requests. The end-result is that faculty sometimes do not feel they are "getting their money's worth" out of DCS services despite staff spending long hours juggling client requests along with back-room needs.

Contracts

Support contracts were designed to make the provision of support easier and reduce barriers between support staff and clients needing support. However, they are an imperfect mechanism; the actual work required to support a client often does not correlate well with the level of support contract. Some contracts require relatively little staff time to fulfill, while others at the same level require substantially more. Clients who demand less feel they pay a disproportionate share of the cost of DCS services.

Growing "Behind the Scenes" Demands

The computing scene has been changing rapidly on campus, and in the rest of the world. The complexity of computing installations is increasing due to advanced software, hardware and network systems that all must interoperate. Due to heavy competition, vendors are rushing products to market without sufficient testing. DCS has to test new software versions, new hardware, and new configurations behind the scenes to limit the impact of technical problems to its clients.

At the same time, a large amount of world wide attention is being paid to the development and dissemination of flaws in computing operating systems that allow unauthorized access. This requires the allocation of a significant amount of DCS and other computing support staff resources to keeping operating systems updated with the latest vendor security patches.

Another example of less visible demands is the Year 2000 problem. DCS has been tracking the developments in each vendor's support of Y2K compliant operating systems. Y2K patches and new versions of each vendor's operating system are periodically released and must be tested. During the year 1999, over 120 systems require operating system upgrades to ensure they will properly function as the year changes to 2000.

The overall result is an increasing amount of infrastructure to support behind the scenes. This burden is largely invisible to clients, who become aware of it only when there are problems from the lack of sufficient preventative attention. Staff are very concerned about it, as they are fully aware of the potential negative impact of not maintaining infrastructure and security. This makes staff less responsive to client requests, even though they have clients' best interests in mind.

DCS Staff

DCS staff work long hours and are under pressure from many sources to meet conflicting demands. Different staff have different talents and abilities, and approach the workload situation in different ways. Some staff are more willing and able to work long hours than others. When these staff leave or realize they cannot maintain the extra hours, DCS' effective manpower goes down. Staff who work long hours also set expectations that are difficult for those who follow to meet.

It has been a challenge to meet client needs with fixed staffing levels, and there is no mechanism in place to ensure that staff are added when workload increases. Because staff positions are only partially paid for out of contract revenue (about 40%), adding new contracts does not in itself provide enough money to hire more staff.

Growing Costs

In addition to the need for additional staff to keep up with growing demands, the average salary of technical staff has also risen. DCS contract rates have not been increased in many years so the additional cost of supporting increased DCS workload has been absorbed in the OIT budget. This leaves fewer resources for other activities.

"Too Busy Sawing to Sharpen the Saw"

DCS staff concentrate on the most critical element of their jobs: keeping systems running and secure. There is insufficient additional staff time to spend on other critical, but non-urgent needs. These include developing NT support services, facilitating the support efforts of departmental staff and student assistants, and facilitating everyone's computer use by making available and maintaining documentation. DCS staff do not have sufficient time to track user needs and stay abreast of their current computing and network applications. This makes it difficult for OIT to help the campus efficiently embrace evolving technology and techniques.

DCS and the UCI Support Model

DCS' current "cooperative" support of local support staff should be expanded, as the existence of local staff is encouraged by UCI's computing support model. However, there is little DCS staff time available to devote to training local support staff, finding tools that make their job easier, facilitating the use of temporary support staff (students), making secure Linux distributions, and for similar activities.

DCSLIB

The DCSLIB software library has been an important factor in supporting UCI Unix users for many years. Having a central network-based software library at UCI has made a rich set of software available to more people on campus than would be the case with local software installations. However, some of the underlying assumptions that motivated DCSLIB's design have changed and DCSLIB must evolve to take these changes into account. Disk space is now much less expensive than it once was. Ten years ago there were virtually no local support staff at all; now all schools and many workgroups have local staff. Much Unix software is now available in a format and distribution convenient for local installations; no such distributions were available when DCSLIB was created.

Some clients are increasingly aware of their dependence on DCSLIB, and are asking that the DCSLIB subset they use be installed locally. This could make their systems more reliable when there are network or DCSLIB problems, but will increase the cost of supporting software installations. If local-access is done properly, the increased costs can hopefully be kept to a minimum.

Top of Page

The Plan

OIT plans to take the following actions in response to the issues summarized above. These actions should ensure that DCS continues to meet client needs as they grow and evolve, while at the same time maintaining manageable staff workloads:

  1. Two additional staff positions are planned for DCS: one to improve the level of support for system-specific system administration tasks, and another to increase the support of infrastructure activities that benefit all. The first position would be funded from additional contract revenue; OIT has requested funds from the campus to cover the second position.
  2. DCSLIB service will be provided campus-wide at no fee. DCSLIB is an example of the sort of heavily leveraged service that should be available to everyone on campus, much like the network, E4E and EA. In addition, OIT will look into providing DCSLIB services in new ways, investigating local servers and/or local copies of critical software.
  3. The current pass-through operating system/compiler license fee of $10/month, as well as other software license fees (e.g. Matlab), will not change.
  4. The fees for file system backup service will be decreased about 50%, as analysis indicates this service costs less to provide than anticipated. Backup service is another example of a critical, highly leveraged service that should be made widely available.
  5. DCS is implementing policies and processes that promote communication responsiveness to client requests.
  6. A new contract scheme will be created with the following support categories:
    Cooperative:
    A basic, low-level support service that provides critical security and maintenance needs at a cost low enough to ensure all Unix systems are covered. This is NOT an end-user service -- it provides behind-the-scenes support of systems and local supporters, and nothing else. Major system upgrades are performed by qualified local supporters, with DCS available for advice and assistance.
    Personal Workstation Support:
    A basic support level for systems used as clients' personal desktop workstations, and configured according to DCS hardware and software standards. These systems are used to access the network, do computing and other work, and do not act as E-mail, Web, or other types of network servers. These configurations require less support and their users should be charged accordingly. A list of the specific services included with "Personal Workstation" support will be created to be used as a guideline in setting expectations.
    Server Support:
    Basic support services for systems that are used as E-mail, file, Web or other types of network servers, are used by more than a few users, or utilize nonstandard configurations. Systems configured this way typically require more staff attention due to multiple users, greater complexity of software interoperation, and greater security implications. A list of the specific services included with "Server" support will be created to be used as a guideline in setting service expectations.
    Tailored:
    A support service much like DCS standard services, but tailored to specific client needs. This service will be used to support all above-nominal client needs with fees set according to the expected amount of staff time required to meet them. It will allow the provision of services that DCS staff feel they can provide, but were outside of the old contract structure. Like the other support levels, a list of specific services included with these sorts of contracts will be created. These guidelines will be used as a starting point to allow the DCS manager to develop a working agreement for supplying tailored services, working with the clients, with the help and acceptance of DCS staff

The new contract scheme will support two models: a) local support staff with central services from DCS (Cooperative support) or b) local and central support from DCS at three levels (Personal Workstation, Server, and Tailored). Dropping the DCSLIB charge will reduce the total fee required for systems under cooperative support, facilitating the local support model. The Personal Workstation fee will be slightly less to reflect the reduced demands placed on DCS, but the other support level fees will be increased to more completely pay for DCS staff time directly associated with end-user systems. This will allow OIT to add DCS staff as client demands increase. Central funding will be used to cover the costs of infrastructure support, where the workload does not grow as quickly as additional client systems are added.

An additional goal will be to move some clients currently on High support, and/or clusters of clients on Regular support, to the new Tailored support level. As these contracts are to be based on full cost recovery of staff time, they will be a much more practical way to meet client expectations than continuing to rely solely on the fixed-price contract structure. Further, unique client needs, such as application and nonstandard operating system support, will be more easily addressed.

The new fees described in the next section are for the 1999/2000 fiscal year. DCS and its clients will need to gain experience with the proposed service structure during 1999/2000 and review rates for 2000/2001. Some feel the new Personal Workstation support configuration will reduce the DCS staff workload required to support those systems, and this price can be reduced further. It is not clear how many of the clients currently on Regular support will be able to move to this support level, and there are other uncertainties as we review contracts that should move to the new Tailored support level. Lastly, the new prices assume that additional funding can be provided from the campus -- if this funding is not available, rates will need to be higher than currently proposed.

Top of Page

New Contract Scheme Details

Cooperative Support

Personal Workstation Support

Server Support:

Tailored support:

Low support:

Backup Services:

Top of Page

DCS Communication and Response Improvements

In addition to making more staff time available by adding two new staff, there are several other steps DCS will take to improve responsiveness:

Call Tracking: DCS will improve its call tracking procedures to ensure that priorities are adhered to group-wide. For example, high-priority requests will be handled even if the primary support staff assigned to the client is unavailable. This is similar to the way urgent requests are handled now. A DCS staff member will monitor the DCS@UCI.EDU service address to ensure staff are located to field high-priority requests.

Group Telephone Line: DCS is reestablishing a method of contacting DCS "staff on-duty" on the telephone, using a DCS group telephone line (824-8383). The group telephone line will be manned during normal business hours. Currently, DCS clients must call the staff member assigned to them, who may be unavailable. Clients can call the general OIT help desk to report urgent matters, but this is less effective than contacting DCS directly.

Responsiveness Policy: A new "responsiveness policy" is being implemented with a goal of 2 business hour response to all requests to the group telephone line (824-8383), and 1 business day response for E-mail requests to the DCS@UCI.EDU group E-mail address. Where as staff may be unable to drop what they are doing and handle new requests immediately, they will touch base to determine the urgency of requests, and give clients some idea of when requests will be handled. An additional goal will be to handle simple questions without delay.

Behind-the-scenes Efforts: DCS will continue its efforts to share information about what goes on in the background by providing documentation, arranging presentations, and providing opportunities for Question and Answer sessions. Although clients are often too busy to learn the full story of system support demands, they must be generally aware of the work done on their behalf so they can support the associated costs.

Client Survey: We would appreciate it if you could take a moment to let us know how we are doing. Your comments are important to us, Thank You.

Top of Page