George K. Thiruvathukal

selected publications

1. IEEE Conference

   A Survey of Methods for Low-Power Deep Learning and Computer Vision

   Goel, Abhinav, Tung, Caleb, Lu, Yung-Hsiang, and Thiruvathukal, George K.

   In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT) 2020

   Abs HTML

   Deep neural networks (DNNs) are successful in many computer vision tasks. However, the most accurate DNNs require millions of parameters and operations, making them energy, computation and memory intensive. This impedes the deployment of large DNNs in low-power devices with limited compute resources. Recent research improves DNN models by reducing the memory requirement, energy consumption, and number of operations without significantly decreasing the accuracy. This paper surveys the progress of low-power deep learning and computer vision, specifically in regards to inference, and discusses the methods for compacting and accelerating DNN models. The techniques can be divided into four major categories: (1) parameter quantization and pruning, (2) compressed convolutional filters and matrix factorization, (3) network architecture search, and (4) knowledge distillation. We analyze the accuracy, advantages, disadvantages, and potential solutions to the problems with the techniques in each category. We also discuss new evaluation metrics as a guideline for future research.

2. IEEE Conference

   FLIC: A Distributed Fog Cache for City-Scale Applications

   West, Jack, Klingensmith, Neil, and Thiruvathukal, George K.

   In 2020 IEEE International Conference on Fog Computing (ICFC) 2020

   Abs HTML

   We present FLIC, a distributed software data caching framework for fogs that reduces network traffic and latency. FLIC is targeted toward city-scale deployments of cooperative IoT devices in which each node gathers and shares data with surrounding devices. As machine learning and other data processing techniques that require large volumes of training data are ported to low-cost and low-power IoT systems, we expect that data analysis will be moved away from the cloud. Separation from the cloud will reduce reliance on power-hungry centralized cloud-based infrastructure. However, city-scale deployments of cooperative IoT devices often connect to the Internet with cellular service, in which service charges are proportional to network usage. IoT system architects must be clever in order to keep costs down in these scenarios. To reduce the network bandwidth required to operate city-scale deployments of cooperative IoT systems, FLIC implements a distributed cache on the IoT nodes in the fog. FLIC allows the IoT network to share its data without repetitively interacting with a simple cloud storage service, reducing calls out to a backing store. Our results displayed a less than 2% miss rate on reads. Thus, allowing for only 5% of requests needing the backing store. We were also able to achieve more than 50% reduction in bytes transmitted per second.

3. IEEE Conference

   Crowdsourcing Detection of Sampling Biases in Image Datasets

   Hu, Xiao, Wang, Haobo, Vegesana, Anirudh, Dube, Somesh, Yu, Kaiwen, Kao, Gore, Chen, Shuo-Han, Lu, Yung-Hsiang, Thiruvathukal, George K., and Yin, Ming

   In Proceedings of The Web Conference 2020 2020

   Abs HTML

   Despite many exciting innovations in computer vision, recent studies reveal a number of risks in existing computer vision systems, suggesting results of such systems may be unfair and untrustworthy. Many of these risks can be partly attributed to the use of a training image dataset that exhibits sampling biases and thus does not accurately reflect the real visual world. Being able to detect potential sampling biases in the visual dataset prior to model development is thus essential for mitigating the fairness and trustworthy concerns in computer vision. In this paper, we propose a three-step crowdsourcing workflow to get humans into the loop for facilitating bias discovery in image datasets. Through two sets of evaluation studies, we find that the proposed workflow can effectively organize the crowd to detect sampling biases in both datasets that are artificially created with designed biases and real-world image datasets that are widely used in computer vision research and system development.

4. Frontiers Journal

   A Computational Theory for the Emergence of Grammatical Categories in Cortical Dynamics

   Dematties, Dario, Rizzi, Silvio, Thiruvathukal, George K., Pérez, Mauricio David, Wainselboim, Alejandro, and Zanutto, B. Silvano

   Frontiers in Neural Circuits 2020

   Abs HTML PDF

   A general agreement in psycholinguistics claims that syntax and meaning are unified precisely and very quickly during online sentence processing. Although several theories have advanced arguments regarding the neurocomputational bases of this phenomenon, we argue that these theories could potentially benefit by including neurophysiological data concerning cortical dynamics constraints in brain tissue. In addition, some theories promote the integration of complex optimization methods in neural tissue. In this paper we attempt to fill these gaps introducing a computational model inspired in the dynamics of cortical tissue. In our modeling approach, proximal afferent dendrites produce stochastic cellular activations, while distal dendritic branches–on the other hand–contribute independently to somatic depolarization by means of dendritic spikes, and finally, prediction failures produce massive firing events preventing formation of sparse distributed representations. The model presented in this paper combines semantic and coarse-grained syntactic constraints for each word in a sentence context until grammatically related word function discrimination emerges spontaneously by the sole correlation of lexical information from different sources without applying complex optimization methods. By means of support vector machine techniques, we show that the sparse activation features returned by our approach are well suited—bootstrapping from the features returned by Word Embedding mechanisms—to accomplish grammatical function classification of individual words in a sentence. In this way we develop a biologically guided computational explanation for linguistically relevant unification processes in cortex which connects psycholinguistics to neurobiological accounts of language. We also claim that the computational hypotheses established in this research could foster future work on biologically-inspired learning algorithms for natural language processing applications.

5. Tech Report

   Observing Responses to the COVID-19 Pandemic using Worldwide Network Cameras

   Ghodgaonkar, Isha, Goel, Abhinav, Bordwell, Fischer, Tung, Caleb, Aghajanzadeh, Sara, Curran, Noah, Chen, Ryan, Yu, Kaiwen, Mahapatra, Sneha, Banna, Vishnu, Kao, Gore, Lee, Kate, Hu, Xiao, Eliopolous, Nick, Chinnakotla, Akhil, Rijhwani, Damini, Kim, Ashley, Chakraborty, Aditya, Ward, Mark Daniel, Lu, Yung-Hsiang, and Thiruvathukal, George K.

   2020

   Abs HTML PDF

   COVID-19 has resulted in a worldwide pandemic, leading to "lockdown" policies and social distancing. The pandemic has profoundly changed the world. Traditional methods for observing these historical events are difficult because sending reporters to areas with many infected people can put the reporters’ lives in danger. New technologies are needed for safely observing responses to these policies. This paper reports using thousands of network cameras deployed worldwide for the purpose of witnessing activities in response to the policies. The network cameras can continuously provide real-time visual data (image and video) without human efforts. Thus, network cameras can be utilized to observe activities without risking the lives of reporters. This paper describes a project that uses network cameras to observe responses to governments’ policies during the COVID-19 pandemic (March to April in 2020). The project discovers over 30,000 network cameras deployed in 110 countries. A set of computer tools are created to collect visual data from network cameras continuously during the pandemic. This paper describes the methods to discover network cameras on the Internet, the methods to collect and manage data, and preliminary results of data analysis. This project can be the foundation for observing the possible "second wave" in fall 2020. The data may be used for post-pandemic analysis by sociologists, public health experts, and meteorologists.

6. IEEE Conference

   Exercises Integrating High School Mathematics with Robot Motion Planning

   Greenberg, Ronald I., and Thiruvathukal, George K.

   In 2019 IEEE Frontiers in Education Conference (FIE) 2019

   Abs HTML PDF

   This Innovative Practice Work in Progress presents progress in developing exercises for high school students incorporating level-appropriate mathematics into robotics activities. We assume mathematical foundations ranging from algebra to precalculus, whereas most prior work on integrating mathematics into robotics uses only very elementary mathematical reasoning or, at the other extreme, is comprised of technical papers or books using calculus and other advanced mathematics. The exercises suggested are relevant to any differential-drive robot, which is an appropriate model for many different varieties of educational robots. They guide students towards comparing a variety of natural navigational strategies making use of typical movement primitives. The exercises align with Common Core State Standards for Mathematics.

7. PLOS ONE

   Phonetic Acquisition in Cortical Dynamics, a Computational Approach

   Dematties, Dario, Rizzi, Silvio B., Thiruvathukal, George K., Wainselboim, Alejandro, and Zanutto, B. Silvano

   PLOS ONE 2019

   Abs HTML PDF

   Many computational theories have been developed to improve artificial phonetic classification performance from linguistic auditory streams. However, less attention has been given to psycholinguistic data and neurophysiological features recently found in cortical tissue. We focus on a context in which basic linguistic units–such as phonemes–are extracted and robustly classified by humans and other animals from complex acoustic streams in speech data. We are especially motivated by the fact that 8-month-old human infants can accomplish segmentation of words from fluent audio streams based exclusively on the statistical relationships between neighboring speech sounds without any kind of supervision. In this paper, we introduce a biologically inspired and fully unsupervised neurocomputational approach that incorporates key neurophysiological and anatomical cortical properties, including columnar organization, spontaneous micro-columnar formation, adaptation to contextual activations and Sparse Distributed Representations (SDRs) produced by means of partial N-Methyl-D-aspartic acid (NMDA) depolarization. Its feature abstraction capabilities show promising phonetic invariance and generalization attributes. Our model improves the performance of a Support Vector Machine (SVM) classifier for monosyllabic, disyllabic and trisyllabic word classification tasks in the presence of environmental disturbances such as white noise, reverberation, and pitch and voice variations. Furthermore, our approach emphasizes potential self-organizing cortical principles achieving improvement without any kind of optimization guidance which could minimize hypothetical loss functions by means of–for example–backpropagation. Thus, our computational model outperforms multiresolution spectro-temporal auditory feature representations using only the statistical sequential structure immerse in the phonotactic rules of the input stream.

8. IEEE Magazine

   See the World Through Network Cameras

   Lu, Y., Thiruvathukal, G. K., Kaseb, A. S., Gauen, K., Rijhwani, D., Dailey, R., Malik, D., Huang, Y., Aghajanzadeh, S., and Guo, M. M.

   Computer 2019

   Abs HTML PDF

   Millions of network cameras have been deployed worldwide. Real-time data from many network cameras can offer instant views of multiple locations for many applications. We describe the real-time data available from these cameras and potential applications.

9. IEEE Conference

   A Survey of Software Metric Use in Research Software Development

   Eisty, Nasir U., Thiruvathukal, George K., and Carver, Jeffrey C.

   In 2018 IEEE 14th International Conference on e-Science (e-Science) 2018

   Abs HTML PDF

   Background: Breakthroughs in research increasingly depend on complex software libraries, tools, and applications aimed at supporting specific science, engineering, business, or humanities disciplines. The complexity and criticality of this software motivate the need for ensuring quality and reliability. Software metrics are a key tool for assessing, measuring, and understanding software quality and reliability. Aims: The goal of this work is to better understand how research software developers use traditional software engineering concepts, like metrics, to support and evaluate both the software and the software development process. One key aspect of this goal is to identify how the set of metrics relevant to research software corresponds to the metrics commonly used in traditional software engineering. Method: We surveyed research software developers to gather information about their knowledge and use of code metrics and software process metrics. We also analyzed the influence of demographics (project size, development role, and development stage) on these metrics. Results: The survey results, from 129 respondents, indicate that respondents have a general knowledge of metrics. However, their knowledge of specific SE metrics is lacking, their use even more limited. The most used metrics relate to performance and testing. Even though code complexity often poses a significant challenge to research software development, respondents did not indicate much use of code metrics. Conclusions: Research software developers appear to be interested and see some value in software metrics but may be encountering roadblocks when trying to use them. Further study is needed to determine the extent to which these metrics could provide value in continuous process improvement.