مقدار تقریبی برنامه ریزی پویا برای تخصیص کانال ارتباط پایین در شبکه های سلولی فمتو شناختی

Both femtocells and cognitive radio (CR) are envisioned as promising technologies for the NeXt Generation (xG) cellular networks. Cognitive femtocell networks (CogFem) incorporate CR technology into femtocell deployment to reduce its demand for more spectrum bands, thereby improving the spectrum utilization. In this paper, we focus on the channel allocation problem in CogFem, and formulate it as a stochastic dynamic programming (SDP) problem aiming at optimizing the long-term cumulative system throughput of individual femtocells. However, the multi-dimensional state variables resulted from complex exogenous stochastic information make the SDP problem computationally intractable using standard value iteration algorithms. To address this issue, we propose an approximate dynamic programming (ADP) algorithm in pursuit of an approximate solution to the SDP problem. The proposed ADP algorithm relies on an efficient value function approximation (VFA) architecture that we design and a stochastic gradient learning strategy to function, enabling each femtocell to learn and improve its own channel allocation policy. The algorithm is computationally attractive for large-scale downlink channel allocation problems in CogFem since its time complexity does not grow exponentially with the number of femtocells. Simulation results have shown that the proposed ADP algorithm exhibits great advantages: (1) it is feasible for online implementation with a fair rate of convergence and adaptability to both long-term and short-term network dynamics; and (2) it produces high-quality solutions fast, reaching approximately 80% of the upper bounds provided by optimal backward dynamic programming (DP) solutions to a set of deterministic counterparts of the formulated SDP problem.

The demand for higher data rates and larger cell capacity has triggered an extensive research effort on the development of the xG cellular networks. One recent progression has come up with the concept of femtocells, which are low-power, short-range, and low-cost plug-and-play home base-stations providing ubiquitous connectivity to macrocell networks via a broadband connection such as digital subscriber line (DSL) or cable modem [1] and [2]. The main merit of the femtocell concept is threefold: (1) Low-power transmission in femtocells not only saves tranmission power, but mitigates inter-cell interference as well. A study in [1] has shown that the transmission power can be saved up to 34 dB and 77 dB with a path-loss propagation model where the indoor path-loss exponents are 4 and 2, respectively. (2) Due to transmitter-to-receiver proximity, femtocells improve cell capacity via more spatial reuse, thus providing higher indoor throughput, which accounts for nearly 80% of mobile data usage according to the most recent network analytics by Cisco [3]. (3) Femtocell deployments spare the need for more macro-base-stations (MBSs), thereby reducing the operating cost for network operators [1]. The problem of channel allocation rises in femtocell deployment, which can be challenging for the following two reasons. First, compared to location-aware access point (AP) placement in macrocell networks, femtocells are deployed by users in a more randomized and autonomous fashion; Second, the number of small-sized femtocells can be quite large within the coverage of macrocell networks, especially in densely populated urban areas. Therefore, applying traditional channel allocation methods used in macrocell networks to femtocell deployment requires more spectrum bands, and will result in poor spectrum utilization [4] and [5]. Meanwhile, it has been shown that the licensed spectrum is inefficiently utilized according to the Federal Communications Commission (FCC) investigations [6]. Hence, over the past decade, there has been a rich literature dedicated to the development of CR technology, which aims at improving the spectrum utilization by allowing unlicensed users to access the licensed spectrum in an opportunistic manner [7] and [8]. This enlightened researchers to incorporate CR technology into femtocell networks [4] and [5], where CR-equipped femto mobile stations (FMSs) and femto base-stations (FBSs) can detect and utilize spectrum opportunities from macrocell networks. We refer to this kind of CR-incorporated femtocell networks as CogFem in this study. There are few research attempts on the channel allocation problem in CogFem. In [4] and [5], the authors studied the downlink spectrum sharing problem in CogFem, and employed decomposition techniques to solve the problem. In [9], the authors proposed a novel cognitive WiMAX architecture with femtocells, and developed an optimization framework for location-aware cooperative resource management based on stochastic Lyapunov optimization. In [10], the authors formulated the energy-efficient resource allocation problem in heterogenous CogFem as a Stackelberg game, and proposed a gradient based iteration algorithm for the equilibrium solution. In [11], the authors proposed a joint power control, base-station and channel assignment scheme in CogFem using multiobjective optimization. In this paper, we investigate the downlink channel allocation problem in overlay CogFem to maximize the throughput of individual femtocells. We adopt stochastic and approximate dynamic programming as our methodologies. On one hand, SDP [12] (see Appendix A) presents a very flexible framework to handle multitude problems with uncertainties; on the other hand, ADP (see Appendix B) has been emerging as a powerful technique for solving complex and large dynamic resource allocation problems in real world [13] and [14]. The principal contributions of this paper can be summarized as follows: (1) We formulate the downlink channel allocation problem in CogFem as a SDP problem, which aims at optimizing the long-term cumulative system throughput while capturing the system dynamics and the characteristics of complex exogenous stochastic information. The SDP formulation requires no knowledge of system state transition probabilities. (2) We propose an ADP algorithm in pursuit of an approximate solution to the formulated SDP problem through reinforcement learning. The algorithm relies on two key enablers to function efficiently: a designed VFA architecture and a stochastic gradient learning strategy. In addition, it ensures the scalability of large-scale downlink channel allocation problems in CogFem since its time complexity does not grow exponentially with the number of femtocells, and is thus computationally attractive. (3) We conduct simulation studies to analyze the feasibility of implementing the ADP algorithm online, and to evaluate its solution quality. Numerical analyses demonstrate that the proposed ADP algorithm is adaptive to both long-term and short-term network dynamics with a fair rate of convergence. In addition, it generates high-quality solutions fast, reaching about 80% of the upper bounds provided by optimal DP solutions to deterministic counterparts of the formulated SDP problem. The remainder of this paper is organized as follows. We summarize the system assumptions, and introduce the system model in Section 2. In Section 3, we formulate the downlink channel allocation problem in CogFem as a SDP problem. Section 4 details the dynamic programming approximation architecture and the proposed ADP algorithm. In Section 6, we evaluate the solution quality of the proposed ADP algorithm through simulation studies. Section 7 concludes the paper.

In this paper, we investigated the downlink channel allocation problem in cognitive femtocell networks. We first formulated the problem as a SDP problem over an infinite horizon. The advantage of this formulation lies in optimizing the long-term cumulative system throughput while capturing the system dynamics and the characteristics of all exogenous stochastic information. Moreover, it does not require access to system state transition probabilities. As an endeavor to overcome the computational intractability incurred by the multi-dimensional state variables, we then proposed an ADP algorithm to solve the SDP problem approximately. The ADP algorithm employs an efficient VFA architecture that we designed, along with a stochastic gradient learning strategy, enabling individual femtocells to learn and improve their own channel allocation policies. The proposed algorithm is computationally attractive for large-scale downlink channel allocation problems in CogFem since its time complexity does not grow exponentially with the number of femtocells. Simulation results have demonstrated that the proposed ADP algorithm is feasible for online implementation with a fair rate of convergence, and is adaptive to both long-term and short-term network dynamics. In addition, the algorithm produces high-quality solutions fast, reaching about 80% of the upper bounds provided by optimal deterministic counterparts of the formulated SDP problem. In our future work, we will focus on addressing the following issues: (1) dynamic channel allocation that guarantees QoS requirements from real-time applications (e.g., end-to-end delay, jitter); (2) in the presence of feedback delay and quantization errors, the impact that imperfect CSI at FBSs would have on the system performance; and (3) power consumption and system resource usage of self-powered FBSs. We also plan on investigating joint adaptive power control, spectrum partitioning and channel allocation schemes.