During the last years, the amount of data has skyrocketed. As a consequence, the data has become more expensive to store than to generate. The storage needs for astronomical data are also following this trend. Storage systems in Astronomy contain redundant copies of data such as identical files or within sub-file regions. We propose the use of the Hadoop Distributed and Deduplicated File System (HD2FS) in Astronomy. HD2FS is a deduplication storage system that was created to improve data storage capacity and efficiency in distributed file systems without compromising Input/Output performance. HD2FS can be developed by modifying existing storage system environments such as the Hadoop Distributed File System. By taking advantage of deduplication technology, we can better manage the underlying redundancy of data in astronomy and reduce the space needed to store these files in the file systems, thus allowing for more capacity per volume.

Based on pathwise duality constructions, several new results on truncated queues and storage systems of the G/M/1 type are derived by transforming the workload (content) processes into certain ‘dual’ M/G/1-type processes. We consider queueing systems in which (a) any service requirement that would increase the total workload beyond the capacity is truncated so as to keep the associated sojourn time below a certain constant, or (b) new arrivals do not enter the system if they have to wait more than one time unit in line. For these systems, we derive the steady-state distributions of the workload and the numbers of customers present in the systems as well as the distributions of the lengths of busy and idle periods. Moreover, we use the duality approach to study finite capacity storage systems with general state-dependent outflow rates. Here our duality leads to a Markovian finite storage system with state-dependent jump sizes whose content level process can be analyzed using level crossing techniques. We also derive a connection between the steady-state densities of the non-Markovian continuous-time content level process of the G/M/1 finite storage system with state-dependent outflow rule and the corresponding embedded sequence of peak points (local maxima).

The problem of controlling input to a stochastic input-output system by accepting or rejecting arriving customers is analyzed as a semi-Markov decision process. Included as special cases are a GI/G/1 model and models with compound input and/or output processes, as well as several previously studied queueing-control models. We establish monotonicity of socially and individually optimal acceptance policies and the more restrictive nature of the former, with random rewards for acceptance and both customer-oriented and system-oriented non-linear waiting costs. Distinctive features of our analysis are (i) that it allows dependent interarrival times and (ii) that the monotonicity proofs do not rely on the standard concavity-preservation arguments.

In this paper we treat the general version of the semi-Markov storage model, introduced first by Senturia and Puri: transitions in the state of the system occur at a discrete sequence of time points, described by a two-state semi-Markov process. An input occurs at an instant of transition to state 1 and a demand for release occurs at an instant of transition to state 2.

Assuming general distributions for all the variables involved, we show that the dam contents just after the nth input converges properly in distribution as n →∞ under conditions of stability; likewise that after the nth demand. We also show that the demand lost due to shortage of stock, accumulated over instants of demand as well as over time, obeys a strong law and a central limit theorem.

In a single-shelf library of N books we suppose that books are returned by placing them at one end of the shelf. The probability of selecting each book is assumed to be known, and only one book is removed at a time. The N! arrangements of the books are considered as states of an ergodic Markov chain for which we find the stationary distribution.