In MT evaluation, pairwise comparisons are conducted to identify the better system. In conducting the comparison, the experimenter must allocate a budget to collect Direct Assessment (DA) judgments. We provide a cost effective way to spend the budget, but show that typical budget sizes often do not allow for solid comparison. Taking the perspective that the basis of solid comparison is in achieving statistical significance, we study the power (rate of achieving significance) on a large collection of pairwise DA comparisons. Due to the nature of statistical estimation, power is low for differentiating less than 1-2 DA points, and to achieve a notable increase in power requires at least 2-3x more samples. Applying variance reduction alone will not yield these gains, so we must face the reality of undetectable differences and spending increases. In this context, we propose interim testing, an "early stopping" collection procedure that yields more power per judgment collected, which adaptively focuses the budget on pairs that are borderline significant. Interim testing can achieve up to a 27% efficiency gain when spending 3x the current budget, or 18% savings at the current evaluation power.

We take a systematic look at the problem of storing whole files in a cache with limited capacity in the context of optimistic learning, where the caching policy has access to a prediction oracle (provided by, e.g., a Neural Network). The successive file requests are assumed to be generated by an adversary, and no assumption is made on the accuracy of the oracle. In this setting, we provide a universal lower bound for prediction-assisted online caching and proceed to design a suite of policies with a range of performance-complexity trade-offs. All proposed policies offer sublinear regret bounds commensurate with the accuracy of the oracle. Our results substantially improve upon all recently-proposed online caching policies, which, being unable to exploit the oracle predictions, offer only $O(\sqrt{T})$ regret. In this pursuit, we design, to the best of our knowledge, the first comprehensive optimistic Follow-the-Perturbed leader policy, which generalizes beyond the caching problem. We also study the problem of caching files with different sizes and the bipartite network caching problem. Finally, we evaluate the efficacy of the proposed policies through extensive numerical experiments using real-world traces.

SOTA decentralized SGD algorithms can overcome the bandwidth bottleneck at the parameter server by using communication collectives like Ring All-Reduce for synchronization. While the parameter updates in distributed SGD may happen asynchronously there is still a synchronization barrier to make sure that the local training epoch at every learner is complete before the learners can advance to the next epoch. The delays in waiting for the slowest learners(stragglers) remain to be a problem in the synchronization steps of these state-of-the-art decentralized frameworks. In this paper, we propose the (de)centralized Non-blocking SGD (Non-blocking SGD) which can address the straggler problem in a heterogeneous environment. The main idea of Non-blocking SGD is to split the original batch into mini-batches, then accumulate the gradients and update the model based on finished mini-batches. The Non-blocking idea can be implemented using decentralized algorithms including Ring All-reduce, D-PSGD, and MATCHA to solve the straggler problem. Moreover, using gradient accumulation to update the model also guarantees convergence and avoids gradient staleness. Run-time analysis with random straggler delays and computational efficiency/throughput of devices is also presented to show the advantage of Non-blocking SGD. Experiments on a suite of datasets and deep learning networks validate the theoretical analyses and demonstrate that Non-blocking SGD speeds up the training and fastens the convergence. Compared with the state-of-the-art decentralized asynchronous algorithms like D-PSGD and MACHA, Non-blocking SGD takes up to 2x fewer time to reach the same training loss in a heterogeneous environment.

Fairness-aware mining of massive data streams is a growing and challenging concern in the contemporary domain of machine learning. Many stream learning algorithms are used to replace humans at critical decision-making points e.g., hiring staff, assessing credit risk, etc. This calls for handling massive incoming information with minimum response delay while ensuring fair and high quality decisions. Recent discrimination-aware learning methods are optimized based on overall accuracy. However, the overall accuracy is biased in favor of the majority class; therefore, state-of-the-art methods mainly diminish discrimination by partially or completely ignoring the minority class. In this context, we propose a novel adaptation of Na\"ive Bayes to mitigate discrimination embedded in the streams while maintaining high predictive performance for both the majority and minority classes. Our proposed algorithm is simple, fast, and attains multi-objective optimization goals. To handle class imbalance and concept drifts, a dynamic instance weighting module is proposed, which gives more importance to recent instances and less importance to obsolete instances based on their membership in minority or majority class. We conducted experiments on a range of streaming and static datasets and deduced that our proposed methodology outperforms existing state-of-the-art fairness-aware methods in terms of both discrimination score and balanced accuracy.

Are passionate about contributing to solutions that benefit science and business at the same time; Are able to communicate with university and business stakeholders; Have knowledge of several of the following techniques: mathematical programming, dynamic programming, reinforcement learning, supervised learning, simulation, business analytics, heuristics, etc.; Can code in one or more of the following programming languages: Python, Java, C, Delphi, Matlab, and R; Have, or will shortly acquire, an MSc degree in Industrial Engineering, Operations Research, Applied Mathematics, or related programme; Possess excellent communication skills and are proficient in English.

All the lecture and lab material is free forever. To be among the first to hear about future iterations of the course, simply enter your email below, follow us on Twitter, or subscribe to our YouTube channel. We released lecture videos on Mondays at 6pm Pacific and lab videos on Wednesdays at 6pm Pacific on YouTube. We review some prerequisites -- the DNN architectures we'll be using and basic model training with PyTorch -- and introduce PyTorch Lightning. We review the purpose of the course and consider when it's a good (or bad!) idea to use ML.

The pandemic has jump-started the transformation of an educational system that has been known to be too slow to evolve, historically. With COVID-19 shutting down schools, over 1.2 billion children were forced out of their classrooms.

In the second season of BBC mystery thriller The Capture, deepfakes threaten the future of democracy and UK national security. In a dystopia set in present day London, hackers use AI to insert these highly realistic false images and videos of people into live news broadcasts to destroy the careers of politicians. But my team's research has shown how difficult it is to create convincing deepfakes in reality. In fact, technology and creative professionals have started collaborating on solutions to help people spot bogus videos of politicians and celebrities. We stand a decent chance of staying one step ahead of fraudsters.

European Leadership University (ELU) has received it educational license in 2015 with the official decree and approval by the Ministry of Education and Culture in Turkish Republic of Northern Cyprus. We have received Institutional accreditation in 2016 and achieved full programme accreditation in 2017 by the Higher Education Planning, Evaluation, Accreditation and Coordination Council, Nicosia which is a member of The European Association for Quality Assurance in Higher Education (ENQA), the umbrella organisation for recognised government accreditation agencies in the European Higher Education Area (EHEA).

Artificial intelligence is expected to have a huge impact on PGDM students, from handling job migration to Robo-educators. Recently, artificial intelligence has moved beyond science fiction and will disrupt how we live our lives. Business is being impacted left, right, and focused. Add to that the extended 40% expansion in labor efficiency from computer-based intelligence use, and the 61% of business experts who say AI and machine learning are their association's most critical information drive. Artificial intelligence is crawling into the daily existence of the present PGDM understudy as well, and occasionally in astonishing ways.