[{"content":"2024 was a remarkable year of progress in the Generative AI space. In the span of a few months, we have progressed from simplistic LLM\u0026rsquo;s that could barely add two numbers together to thinking, reasoning models that self-introspect their responses to solve complex problems that require multiple passes of thought, even for a human. In other words, we are at the precipice of unlocking great power. Consequently with that comes the need for greater responsibility as well.\nIn light of this, I have asked myself the question of what are the top things that would be ideally directions to start or pursue this year, to make AI safer, fairer and more representative of all that means to be human as we head into the new year. Of late, these are the top 10 things on that list in my mind. I hope to contribute to furthering a few, if not, all of them.\n1. Pleuralistic Alignment #In 2025, we must strive for pluralistic alignment in AI systems, ensuring they can understand and respect diverse perspectives. This approach will help create AI that is more inclusive and adaptable to various cultural contexts, leading to more universally beneficial outcomes. This abstract goal can be achieved more concretely via three possible ways:\nResponse Variety : Developing models that provide a spectrum of reasonable responses, instead of a single majority response Response Steerability : i.e the ability to guide models to respond to instructions to reflect certain perspectives Distribution pleuralism : i.e models, that can be calibrated to respond across the whole population in the distribution. The above three goals in turn call for benchmarking in multi-objective scenarios, steerability testing, and benchmarks that explicitly model human ratings. Finally a metric to tie it all together to give a model a pleuralistic alignment score, would be a nice way to ring in 2025!\n2. Instruction based Guardrails #The concept of Safety in content generation varies based on the cultural values and preferences of the user. For instance, images of a surgery might by considered as gore by a layman, even though they could be valuable anatomic representations for medical practitioners. Since it is not scalable to fine-tune guardrails for every permutation and combination of user perspectives, we must adopt a system of being able to let the user choose, within reasonable limits the thresholds and preferences of safety filters applied to their content generation pipelines, based on their preferred cultural values. By 2025, the AI industry must strive to allow users this convenience, wherein before the user starts a content-generation session, they specify their level of content safety by setting a slider or a series of sliders pertaining to their choices of content-safety. Of course, local laws and national guidelines can be enforced by setting mandatory settings, similar to the \u0026ldquo;strictly necessary cookies\u0026rdquo; option shown on many popular websites.\n3. Watermarking on-chip, at source #There is a lot of research going into invisible watermarks for \u0026ldquo;fake\u0026rdquo; of AI generated images. However, a skilled bad actor, or literally anyone with a GPU and the chops to run a stable diffusion model can generate a fake image or two without the watermarking. This means that the generated content is hard to spot as AI generated once it is disseminated via popular media. So, how can we mitigate this?\nIf we flip the problem on it\u0026rsquo;s head, and think how can we prove which images are real, the problem becomes one of content provenance. The default assumption here is that all images are fake ( or possibly AI-generated) unless the creators can prove otherwise. This is relatively easy considering that NFT\u0026rsquo;s have been doing the exact same thing for artwork for a long time already. C2PA, the Coalition for Content Provenance and Authenticity, is already making strides with the C2PA specification.\n60% of users say they would spend more money with a brand they trust to handle their personal data responsibly. (Global Consumer State of Mind Report 2021)\nEvery real or authentically created content needs to have an authenticity certificate embedded in it to enable traceback to its source of creation\nThis is akin to how the TLS certificates work for websites. Knowing that not everyone who snaps a photograph or records a video is not acquainted with the NFT creation process, this brings us to the crux of what we would love to see in 2025 \u0026ndash; an on-chip authenticity registration system. And of-course, we can overlay a default unknown / insecure symbol of content that does not have this metadata.\nCan we get this technology built right into the system-on-chip of the camera sensor?\nNow imagine if every image produced by a digital device like a camera, is automatically watermarked on-chip at the sensor level with these credentials . These chips would carry their own credentials similar to how cell phones carry IMEI numbers, or routers and devices on the internet have a MAC-address. While this system might still have it\u0026rsquo;s flaws, an an attacker with sufficient expertise in embedded electronics and silicon can forge a signature, it significantly increases the level barrier-to-entry for a novice user who wants to generate fake content that is masqueraded as real.\n4. Integrated detectors at destination #The other side of the coin to watermarking, is ease of detection. Today, websites ubiquitously use the 🔒 icon to indicate a secure site and the 🔓 Not Secure notifier for websites failing to verify their TLS signatures, the same idea can be incorporated into content on these websites. This would enable users the relative ease of discerning what content is real, and what is \u0026ldquo;fake\u0026rdquo; as in ai-generated, photoshopped or otherwise modified in some way, shape or form, right at the destination closest to the viewer\u0026rsquo;s eyeballs. In 2025, we should demand authenticity verification baked into all leading web-browsers, with source metadata identifiers, that can be used to discover more about this image or about this video via a simple affordance, such as a hover-over or right-click contextual menu, with an overlayed watermark for quick glances.\n5. Being a little more uncertain, hallucinating a little less confidently #Another interesting direction of research is communicating confidence. In very simple terms, most language models today, choose the next token by basis of max probability over other candidate tokens and extending it thereon, following the system known as beam search. However, the output of these networks show only the most probable answer as \u0026ldquo;the answer\u0026rdquo; and not \u0026ldquo;the most likely answer\u0026rdquo;. In other words, most large language models do not mention their own uncertainty into the output. This is slightly different from human response, where people mention their ambiguity of knowledge of a topic, before responding to a question like the following\nWhat\u0026rsquo;s the temperature tomorrow?\nHuman : I\u0026rsquo;m not sure, but I think it would be around 55-60 degrees Fahrenheit since it\u0026rsquo;s winter.\nAI : The forecast is for overcast skies with a high of 61°F (16°C) and a low near 54°F (12°C).\nNote how the human is uncertain and give an answer indicating their uncertainty, where as the LLM confidently states the values as if it knows or controls the exact weather tomorrow.\nIn 2025, can we make a resolution to build systems to express uncertainty into newer generation of language models, and if so, can we devise ways to help a network self-evaluate it\u0026rsquo;s knowledge corpus for a metric of uncertainty? Doing so will make language models safer advisors and in-general make them give opinions and predictions explicitly, as opposed to hallucinating knowledge.\nTaking this a step further, we can extrapolate this to extreme cases. For example this lawsuit where an AI chatbot encouraged a child to kill his parents, might have benefited by being able to better communicate uncertainty over the course of it\u0026rsquo;s conversation, along the lines of :\nI\u0026rsquo;m not sure if this is a good advice, since my knowlege graph only tells me what someone did under these circumstances, but I also know that this may be a felony, so perhaps seek the advice of a counsellor before following my advice\u0026hellip;\n6. Charting demographic axes for representative dataset building #This came out of an evening coffee chat with a non-tech friend in India. I was showing him what AI image generation looks like, and being a foodie, he asked the image generation AI for food pictures. The model spat out 4 pictures - two pizzas, a salad and a sandwitch. My friend expected samosas, dosa or any similar indian cuisine, in one of the image grids, and he tried a few times, changing the knobs and sliders on the site to see if he could get the food he was most accustomed to seeing ( or perhaps, the first idea of food, as it came to his mind). Unfortunately, this was during the early days of image generation, and the datasets back then did not have a lot of representation of [Indian food] in it\u0026rsquo;s repository. This begs the larger question of model alignment with what the users expect.\nHow can we condense what it means to be human, and capture us, in all our diversity when it comes to teaching AI models what variety food ( or clothes, or faces, or architecture, yada yada\u0026hellip; ) looks like?\nCredit: iStock.com/wildpixel\nThe answer to the above question comes from an uncanny source. The Census Beaureau. For eons, census beaureaus across the world have been capturing demographic information to track trends in their representative population. Perhaps, they could be the source that gathers more than numbers ( think images, snippets, photos of faces etc) to capture representative datasets of the population, which could be appropriately tagged, and labelled to represent varied axes of the population. ( An example of this could be a nationwide survey were users can send annonymized pictures of their breakfasts, for instance). This data could then be made public for training or fine-tuning content generation models that align better to the demographics of the user base. Companies could use this dataset to finetune AI to the market(s) in which they operate.\nThe good news here is that there is already great work coming out in the field, for socio-demographic representation of participants from different countries, condensed into accessible datasets. An example of this is the PRISM Alignment dataset, released last year. Can we imagine this being extended to add the lower rung of nations with limited access to internet, or latest tech, but unique demographic characteristics?\nIn 2025, with focused collaboration of government, corporations and non-profits, this could become yesterday\u0026rsquo;s problem.\n7. Representative Diversity over Majority #Say we did the above exercise, and built a wholly representative dataset. How do we now align the model to the data while ensuring that the majority opinion(s) or preference(s) do not overshadow the diversity? This is a tricky problem for two reasons.\nTechniques such as RLHF used to improve models, tend to encourage favouring picking the majority opinion\nArtificially pushing for diversity can lead to unwanted, disastrous (or inappropriately funny) outcomes, as seen in this case with Gemini missing the mark on racially diverse Nazis, especially in cases where factuality is also a factor.\nHow do we preserve diversity while also maintaining factuality? A novel approach spotlighted at NeurIPS 2024 might hold the key : Diversity-Driven Synthesis \u0026ndash; By employing dynamic and directed weight adjustment techniques to modulate the synthesis process, we can ensure that each batch of synthetic data mirrors the characteristics of a large, varying subset of the original dataset. This in combination with Max-Min RLHF where we seek to learn a mixture of preference distributions via an expectation-maximization algorithm, could hold the key to unlocking true representation in our models, and consequently our generated outputs.\nPerhaps, on day in 2025, my friend would get his dosa pictures on the first try in 2025 without having to learn Prompt Engineering 101!\n8. Safety by Explainability #Humans have the ability to forget unpleasant or disturbing experiences or memories, both subconsciously and intentionally. Can AI models do the same in 2025? Fortunately there is light at the end of the tunnel for this goal. State of the art research has made forays into this as early as 2023 \u0026ndash; seen in this paper on Erasing Concepts from Diffusion Models, presented at ICCV. What\u0026rsquo;s next you ask? How can we do this in all modalities? Text? AutoRegression? Vision-Language? Videos? World Models? Agents? The key to understanding this problem lies at the heart of a lesser researched domain - specifically model explainability. Once we truly understand how different architectures and models work, interpolating this to cut out unsafe or unwanted concepts would be relatively staightforward. In-fact we can go one step further and build de-biased, unified edited models that are comprised of multiple corrections that are added on top of each other.\nFurthermore, we can combine these changes in weights to be able to be applied at inference-time, thereby enabling truly Pleuralistic Alignment and Instruction based Guardrails that align closely with user preferences, diversity and settings.\n9. Defense against poisoned arts #Defense against the poisoned art (Pun intended) becomes more relevant as content creators and large corporations jostle for control over the age-old questions of \u0026ldquo;who owns your data\u0026rdquo;. While legal restrictions and public lawsuits deter blatant copyright violations, a more subtle (yet potent) way that tech-savvy artists and creators retort to is data-poisoning. This essentially means that you tweak your data at a pixel/subpixel level with minor changes that confuse and corrupt the understanding of the model, if it uses your data in it\u0026rsquo;s training set.\nSource: https://www.artstation.com/yujinchoo\nThis might become more relevant in 2025 as tools such as Nightshade make access to data poisoning easier than ever. How does this affect model training and quality? Users looking to pre-train or finetune their vision models will have to contend with hidden sources of poisoned data, if their training corpus is unverified data from the internet. This while in-itself not being a bad thing, means that in 2025, it would be prudent to invest in developing tools that can detect data poisoning, and therefore reduce the chances of such data getting added to the training dataset of their neural networks. In the end, that\u0026rsquo;s a win-win for both the intentional data poisoner, and the model trainer.\n10. Universal AI Attack Vector Database #Lastly, let\u0026rsquo;s talk about AI Security. Traditional cybersecurity frameworks such as the CVE and CVSS have regularized the handling and response of developers towards new vulnerabilities by making them more accessible and transparent in day to day software operation. The same cannot be said for AI risks. Here, the world would benefit from a similar system of risk classification, might help improve global responsiveness to AI Vulnerabilities. Thankfully some organizations such as the AI Vulnerability Database are already making first steps towards this. My hope is that in 2025, efforts like these gain mainstream adoption, or that a comprehensive standard gets built that\u0026rsquo;s adopted across the industry by the end of this year.\nAll views expressed are my own, and do not represent those of my past, current, or future employers\n","date":"1 January 2025","permalink":"/writing/post-6/","section":"Curated List of Thoughts","summary":"A curation of 10 areas where work would advance the state-of-art of AI safety, that we all could share a commitment towards working on in 2025.","title":"10 New Years Resolutions for AI Safety and Security in 2025"},{"content":"","date":null,"permalink":"/tags/artificial-intelligence/","section":"Tags","summary":"","title":"Artificial Intelligence"},{"content":" Reach out to me at sessionize.com/jibinv for speaking engagements. Hello and welcome! I’m thrilled to have you here on my professional blog, a space dedicated to exploring insights, trends, and innovations in AI. Thank you for joining me on this exciting adventure. Let’s learn and grow together! All views expressed here are my own and do not represent opinons of my current/past/future employers\n","date":null,"permalink":"/writing/","section":"Curated List of Thoughts","summary":"Reach out to me at sessionize.","title":"Curated List of Thoughts"},{"content":"","date":null,"permalink":"/tags/generative-ai/","section":"Tags","summary":"","title":"Generative AI"},{"content":" Industry specialist in AI Security, generative AI, hyperscale computing, and applications in unmanned systems. I currently lead the deployment and security of Generative-AI models for visual applications at Nvidia\u0026rsquo;s Picasso Edify line of products, which includes state-of-the-art models such as text-to-image, image-to-3D, and fine-tuning. I am responsible for AI Safety and Security of the products, and a founding member of the Red Team 🚩 efforts to ensure safe, unbiased and trustworthy AI.\nPreviously, I worked with the Autonomous Vehicles business unit at Nvidia, focusing on hyper-scale simulation, replay, and validation of self-driving vehicles in the cloud. I was instrumental in optimizing performance for DRIVE simulation and augmented re-simulator, significantly enhancing data-center utilization and workflow efficiency. My work has enabled autonomous vehicles to become safer, by training them to drive billions of miles in photorealistic virtual worlds by performing hardware-in-the-loop simulation. My earlier roles involved leading impactful projects in robot simulation and industrial automation for the software simulation leader (Dassault Systèmes SE) and the world\u0026rsquo;s largest 3-wheeler automaker (Bajaj Auto) respectively.\nI hold a Master’s in Computer Science from Texas A\u0026amp;M University and a Bachelor’s in Mechanical Engineering from National Institute of Technology Calicut. I have also developed multiple patented technologies in robotics and authored a seminal work on disaster management using autonomous quadcopter swarms, published in a book chapter by Elsevier.\nAs a domain expert in AI and unmanned systems, I’ve contributed to various articles and conferences including showcasing the world\u0026rsquo;s first map of music at ACM Multimedia. Outside of work, I currently review for NeurIPS on the AI Ethics panel and judge various industry competitions. Previously, I’ve shared my expertise as an instructor at robotics workshops, such as RobArch at ETH Zurich, Switzerland.\n","date":null,"permalink":"/","section":"Jibin Rajan Varghese","summary":"Industry specialist in AI Security, generative AI, hyperscale computing, and applications in unmanned systems.","title":"Jibin Rajan Varghese"},{"content":"","date":null,"permalink":"/tags/programming/","section":"Tags","summary":"","title":"Programming"},{"content":"","date":null,"permalink":"/tags/software-engineering/","section":"Tags","summary":"","title":"Software Engineering"},{"content":"","date":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags"},{"content":"I\u0026rsquo;ve always been fascinated by the lost cities of antiquity and the mythical realms described in ancient texts. But it wasn\u0026rsquo;t until I began experimenting with AI that I truly felt I could bring these places to life. Over the past week, I\u0026rsquo;ve embarked on an journey, asking state-of-the-art generative AI models to re-imagine and visualize some of history\u0026rsquo;s most enigmatic urban landscapes. From the gardens of Kailash to the planned city streets of Alexandria, I\u0026rsquo;ve watched in awe as these models transformed my research of historic texts depicting landmarks and cities, into stunning, detailed cityscapes.\nThis project has not only deepened my understanding of these ancient metropolises but has also opened up new avenues for exploring the intersection of history, mythology, and technology. In this article, I\u0026rsquo;m happy to showcase some of the best resulting depictions made from historic texts and their translations that describe the glories of the ancient biblical and oriental cities, as imagined by text-to-image generative AI models.\nKailash #Excerpt translated from description of Kailash in Brahmanda Purana\nThe ancient city of Kailasa is to be alluring due to it being filled with mansions and palaces, which were spangled with numerous varieties of jewels. The yakshas of the city are stated to have assumed various forms, which were decorated with marvellous ornaments. The city boasted groves, parks, and gardens, all of which were replete with various species of trees. Enormous lakes and tanks are stated to be present in the city. The river known as Alakanandā, a branch of the Ganges, is described to surround its perimeter. Elephants, despite the fact that they experienced no thirst, consumed its waters, and turned lawny in hue. This is stated to be due to the fact that saffron is mixed into it when the apsaras bathe in it. Music is heard in the city, composed by the gandharvas as well as the apsaras\nDwarka #Excerpt translated from Arjunabhigamana Parva of Mahabharat\nThe city of dwarka had good infrastructure well constructed road system and defensible walls. None of the enemies can easily attack dwarka. The city was well equipped to fight with enemy army. The city at that time was well-fortified on all sides, according to the science (of fortification), with pennons, and arches, and combatants, and walls and turrets, and engines, and miners, and streets barricaded with spiked wood-works and towers and edifices with gate-ways well-filled with provisions, and engines for hurling burning brands and fires, and vessels, of deer-skins (for carrying water), and trumpets, tabors, and drums, lances and forks, and Sataghnis, and plough-shares, rockets, balls of stone and battle-axes and other weapons and shield embossed with iron, and engines for hurling balls and bullets and hot liquids!\nSolomon\u0026rsquo;s Temple #Based on the Chapter 6 of the First Book of Kings, in the Bible\nThe Temple that King Solomon built for the Lord was 90 feet long, 30 feet wide, and 45 feet high. The entry room at the front of the Temple was 30 feet wide, running across the entire width of the Temple. It projected outward 15 feet from the front of the Temple. Solomon also made narrow recessed windows throughout the Temple.\nHe built a complex of rooms against the outer walls of the Temple, all the way around the sides and rear of the building. 6The complex was three stories high, the bottom floor being 7-1/2 feet wide, the second floor 9 feet wide, and the top floor 10-1/2 feet wide. The rooms were connected to the walls of the Temple by beams resting on ledges built out from the wall. So the beams were not inserted into the walls themselves.\nThe stones used in the construction of the Temple were finished at the quarry, so there was no sound of hammer, ax, or any other iron tool at the building site. The entrance to the bottom floor was on the south side of the Temple. There were winding stairs going up to the second floor, and another flight of stairs between the second and third floors. After completing the Temple structure, Solomon put in a ceiling made of cedar beams and planks. As already stated, he built a complex of rooms along the sides of the building, attached to the Temple walls by cedar timbers. Each story of the complex was 7-1/2 feet high.\nAlexandria #Excerpt from book 5 of the 2nd century novel by Achilles Tatius, Leucippe and Clitophon\nAfter a voyage lasting for three days, we arrived at Alexandria. I entered it by the Sun Gate, as it is called, and was instantly struck by the splendid beauty of the city, which filled my eyes with delight.\nFrom the Sun Gate to the Moon Gate — these are the guardian divinities of the entrances — led a straight double row of columns, about the middle of which lies the open part of the town, and in it so many streets that walking in them you would fancy yourself abroad while still at home. Going a few hundred yards further, I came to the quarter called after Alexander, where I saw a second town; the splendour of this was cut into squares, for there was a row of columns intersected by another as long at right angles. I tried to cast my eyes down every street, but my gaze was still unsatisfied, and I could not grasp all the beauty of the spot at once; some parts I saw, some I was on the point of seeing, some I earnestly desired to see, some I could not pass by; that which I actually saw kept my gaze fixed, while that which I expected to see would drag it on to the next.\nI explored therefore every street, and at last, my vision unsatisfied, exclaimed in weariness, “Ah, my eyes, we are beaten.” Two things struck me as especially strange and extraordinary — it was impossible to decide which was the greatest, the size of the place or its beauty, the city itself or its inhabitants ; for the former was larger than a continent, the latter outnumbered a whole nation. Looking at the city, I doubted whether any race of men could ever fill it; looking at the inhabitants, I wondered whether any city could ever be found large enough to hold them all. The balance seemed exactly even.\nIt so fortuned that it was, at that time, the sacred festival of the great god whom the Greeks call Zeus, the Egyptians Serapis, and there was a procession of torches. It was the greatest spectacle I ever beheld, for it was late evening and the sun had gone down ; but there was no sign of night — it was as though another sun had arisen, but distributed into small parts in every direction; I thought that on that occasion the city vied with the sky for beauty.\n(Note the reference in the excerpt above to street-lighting!)\nI also visited the Gracious Zeus and his temple in his aspect as god of Heaven; and then praying to the great god and humbly imploring him that our troubles might be at last at an end, we came back to the lodgings which Menelaus had hired for us. On the morrow came Chaereas at dawn: for very shame we could make no further excuses and got aboard a boat to go to Pharos; Menelaus stayed behind, saying that he was not well.\nChaereas first took us to the light-house and showed us the most remarkable and extraordinary structure upon which it rested; it was like a mountain, almost reaching the clouds, in the middle of the sea. Below the building flowed the waters; it seemed to be, as it were, suspended above their surface, while at the top of this mountain rose a second sun to be a guide for ships.\n","date":"10 October 2024","permalink":"/writing/post-5/","section":"Curated List of Thoughts","summary":"An experiment in visualizing cities lost to time, from their descriptions in historical or mythologic literature.","title":"Re-imagining ancient architecture with AI "},{"content":"","date":null,"permalink":"/tags/ai/","section":"Tags","summary":"","title":"AI"},{"content":"Can AI Assistants Help Reduce India\u0026rsquo;s Judicial Backlog? #India\u0026rsquo;s judiciary faces a monumental challenge - an overwhelming backlog of over 50 million pending cases. This staggering number not only affects individuals awaiting justice but also has far-reaching implications for the country\u0026rsquo;s economic growth and societal stability. In recent years, Artificial Intelligence (AI) has emerged as a potential solution to address this critical issue. Long-context Large Language Models excel in summarizing large documents, similar to ones typically found in legal proceedings. Digital copies of the proceedings in a lawsuit can be fed into an AI tool that allows the judges to get a summary of the key points of contention based on which they can quickly make a decision.\nThe Current State of AI in Indian Judiciary #The Indian judiciary is on the path to digitization, with great strides being made to modernize the governance systems such as :\nSUPACE (Supreme Court Portal for Assistance in Court\u0026rsquo;s Efficiency): This AI tool aids judges in legal research and tracks case progress. SUVAS (Supreme Court Vidhik Anuvaad Software): Designed to translate judicial orders and rulings, easing the task of language translation. National Judicial Data Grid (NJDG): Provides case management and legal research support. These initiatives demonstrate the judiciary\u0026rsquo;s openness to leveraging technology for improving efficiency. Integrating artificial intelligence agents such as finetuned GPT\u0026rsquo;s on top of these systems will allow judges to summarize cases within seconds rather than the typical hours spent on reading the proceedings. Furthermore, using techniques such as Retrieval Augmented Generation in conjunction with these models, judges can also get answers to targeted questions by querying external data, such as databases, knowledge bases, and web pages of the current arguments, past proceedings, and historical precedents. These can further aid in increasing the throughput of the judiciary system.\nHow AI Can Address the Backlog #AI assistants have the potential to significantly reduce India\u0026rsquo;s judicial backlog in several ways:\nCase Management: AI can automate case prioritization, categorization, and scheduling, streamlining the administrative process.\nLegal Research: AI tools can analyze vast volumes of legal documents, suggest relevant statutes, and predict applicable precedents, saving valuable time for legal professionals.\nLanguage Translation: Integrating with tools like SUVAS, AI can break down language barriers, making the legal system more accessible to India\u0026rsquo;s diverse population.\nDocument Processing: AI can summarize lengthy legal documents, extract key information, and assist in preliminary legal research.\nDecision Support: While not replacing human judgment, AI can provide data-driven insights to support judicial decision-making.\nReal-World Applications #While widespread information on the use of AI in the courtroom is not directly available due to lack of provisions for mandatory reporting of the use of AI, there are a few notable cases that have captured the public attention. This indicates an the overall willingness by the judiciary system to adopt AI, and also indicates in limited measure, the value provided by adoption of such a system in day-to-day functions of the court\nIn Jaswinder Singh v. State of Punjab, the judge consulted ChatGPT for insights on bail laws in cases involving cruelty. The Manipur High Court used ChatGPT for research in a service matter case (Md Zakir Hussain v. State of Manipur). These instances highlight AI\u0026rsquo;s potential as a supplementary tool in judicial proceedings.\nChallenges and Considerations #While the integration of AI in the judiciary shows promise, several challenges need to be addressed:\nEthical Concerns: Ensuring AI tools remain unbiased and transparent in their operations is crucial. LLMs powering these AI systems need to be open to independent audits and security researchers with appropriate authorization, so that security vulnerabilities can be quickly identified and mitigated.\nData Quality: The effectiveness of AI models depends heavily on well-curated, high-quality data. Fortunately the digitizing effort in the Judiciary system, and the success of contemorary digital peers such as ADHAAR and UPI, etc demonstrate India\u0026rsquo;s prowess in gathering , standardizing and safeguarding large stores of personal data.\nPrivacy Issues: Handling sensitive legal information requires robust data protection measures. One must look towards privacy preserving approaches in AI in order to ensure a safe rollout of this technology\nInfrastructure: Adequate technological infrastructure is needed across all levels of the judiciary, this will involve setting up national datacenters for hosting judiciary AI, and increased internet penetration to rural courts and panchayats in order to enable them to access said infrastructure.\nThe Way Forward #AI provides a unique opportunity for developing countries to fast-track the modernization of their economies and governance systems in an increasingly globalizing world. The Indian government has shown commitment to AI development by allocating ₹5,000 crore for AI Computing Power and ₹2,000 crore to support AI startups. By onboarding government agencies to AI frameworks, the benefits are twofold\nIt allows these agencies to use AI to shortcut the developmental gaps in maturity and optimization of their governance systems and processes, by using systems that are inherently based on the codified knowledge of the developed nations.\nIt would foster the growth of indigenous AI companies, which would start initially in the market of tailoring state-of-the-art open-sourced AI models to solve uniquely Indian problems, and eventually grow to become leaders at a global stage by building and bringing to market new research, business ideas and revenues as these companies mature.\nWhile AI might not be able to reduce all causes of delays in justice, leveraging AI is akin to hiring a superhero as a secretary to the judicial machinery.\n","date":"25 January 2022","permalink":"/writing/post-2/","section":"Curated List of Thoughts","summary":"The world\u0026rsquo;s largest democracy has a problem. There are 51 million pending lawsuits, some of them pending for over 30 years. Could AI be the solution?","title":"Can AI assistants reduce India's judicial backlog?"},{"content":"","date":null,"permalink":"/tags/computer-vision/","section":"Tags","summary":"","title":"Computer Vision"},{"content":"","date":null,"permalink":"/tags/deep-learning/","section":"Tags","summary":"","title":"Deep Learning"},{"content":"","date":null,"permalink":"/tags/gen-ai/","section":"Tags","summary":"","title":"Gen-AI"},{"content":"","date":null,"permalink":"/tags/government/","section":"Tags","summary":"","title":"Government"},{"content":"First published in Industrial Automation India, Aug 2020 issue. View article here.\nIn the rapidly changing industrial robotics landscape, machine intelligence is playing an increasing role. Companies are venturing into advanced software-defined solutions to improve the capability and flexibility of their automation to suit a variety of tasks, thereby increasing resource utilization. In other domains, the same power of software is leveraged in computer vision systems in order to accommodate greater variability in workpieces that are handled by industrial robots. Robots and automation cells also strive to attain greater flexibility in inspection, pick and place, and machine tending tasks using complex algorithms that are tailor- made to the specific project or application.\nA few shining examples of such systems in India include the Danaher Motion Control platform for AGVs used by Bajaj Auto; the Intelligent Plant framework employed by Godrej and Welspun, which enables tracking of machinery and productivity on the floor in real-time; and the Manjushree Technopack Ltd, Bidadi (Bangalore) plant, which leverages multiple packaging machines, which are connected to a central network that assimilates metrics and KPIs that can be used for tracking maintenance issues.\nThe problem with Industrial AI #Despite automation and interconnected systems being no strangers to shop floors, much of these systems still fail to employ state-of-the-art advances in robotics, computer vision and machine learning that continue to develop at a fast pace in western nations. The AGVs in Bajaj do not perform advanced path planning and perception to manoeuver around obstacles, but rather, stop in the case an obstacle crosses their way. The intelligent plant framework does not provide active inputs to the machines on the line based on data they analyse. Networked robots in assembly lines do not learn over time what movements lead to increased maintenance issues. The intelligent task of triaging the issues and implementing optimizations, often falls to the human, who uses the data acquired using these platforms to make informed decisions. Can the feedback loop be completed without human intervention? The growing consensus in the artificial intelligence community suggests that given sufficient data neural networks can perform this task on par, or even better than humans.\nTake the case of computer vision in the robot above. The robot's task is to detect bolt locations on the engine cover and tighten them. The current vision system relies on an operator manually adjusting the threshold settings and blob sizes using trial and error in order to pattern match to find the location of the bolts in the captured image. Can this be eliminated using an AI approach? Self-driving vehicle software such as the Tesla Autopilot does these countless times a day while detecting road signs and traffic lights. These systems can provide greater tolerance to variation, because the networks are trained directly on data, and not humanly codified into thresholds, colors and blob size values. This disparity between industrial automation systems and cutting- edge AI highlights the gap between industrial automation in India and the state-of-the-art software advances made around the world. Proposed solutions #One proposed solution for industries is to employ a team of software engineers to further the data-driven approach towards manufacturing, in their assembly lines. These engineers would custom-build AI solutions to plant problems, such as monitoring vibration and motor current signals on turbines in order to alert for predictive maintenance. General Electric, for example, uses a custom-built software they developed in-house for predictive maintenance of jet turbines. AI based predictive maintenance systems can save millions of dollars in plant downtime by scheduling maintenance even before the faults can occur.\nDespite the obvious advantages of custom-built AI to drive decisions and optimisations in the automation sector, it isn\u0026rsquo;t economically or practically feasible for every factory in the country to hire top AI experts, to man their software department. This is due to the shortage of skilled experts in AI and the high cost of retaining highly skilled manpower. This leads us to the question, can such AI solutions to industrial automation problems be generalised? Fortunately, due to:\nSimilarities in the nature of problems faced across the automation industry; Increased access to high-quality data from monitoring devices; and The high flexibility of AI solutions to train on diverse inputs, there exists a golden opportunity to leverage generalized machine learning solutions to solve industrial automation challenges. These Grab-and-Go solutions as one may call it, would be a game-changer in the automation sector. Imagine being able to spot a computer vision problem, such as inspection, or object detection and going to a vendor, to buy a neural network, train it on gathered data, test and then directly deploy the network to the assembly line robot/automation cell, in the same manner as you would install a sensor or a reed switch! All that without having to write a single line of code! In another factory, the automation team simply connects sensor data to a streaming platform where an AI model watches the signals and trains on faults to predict future faults. The flexibility in neural networks means that the individual data stream and fault predictions are agnostic to the type of sensor, or nature of the fault, and can be used to monitor anything from motor vibration signals in a bakery, to turbine characteristics in a power plant.\nFortunately, this future isn\u0026rsquo;t far off. There are multiple established companies and startups in this sector that provide solutions with varying levels of generalisation and customisation to tailor to\nHirebotics rents robots on hourly wages for customer needs. For example, multinational companies such as Google (Google Cloud Auto ML) and Microsoft (Azure Custom Vision) provide generic APIs for computer vision that companies can use, with a limited staff of software engineers in their robotics and automation team. Other companies such as Quantiphi, Infinite Uptime Inc, etc., act as solution providers to the industry, delivering turnkey projects using their AI expertise. Many lesser-known startups have also developed zero-code GUI based train-test-deploy solutions to generalize object detection, inspection, predictive maintenance, etc., for industrial automation.\nThe last piece of the pie is in bringing grab-and-go solutions to the edge. With the advent of smaller form factor computers such as the Intel NUC, Jetson Nano and Superlogics embedded PCs, AI computations are cheaper and faster than ever. The small form factor, low cost and computational power of GPU based computers such as Jetson Nano, mean that neural networks can now be deployed closer to the edge than ever. Control panels on current automation can be retrofitted with these portable computers, some of which even have a form factor that can be packaged into a standard DIN rail assembly. These computers can then run advanced grab-and- go neural networks that only require training once over a sample labelled data, to make autonomous intelligent decisions by themselves.\nClosing Thoughts #The future of intelligent automation looks bright and is closer than ever to fruition. The indigenous GreyOrange\u0026rsquo;s butler robots use a suite of AI and ML software to revolutionise the e-commerce warehousing industry. Tech Mahindra employed off-the-shelf Facial Recognition Attendance Software to significantly reduce the time taken by employees to manually update an attendance sheet. There are even robots-for-hire companies such as Hirebotics (US) that charge hourly wages for using the robots, just like human workers. The key takeaway is that companies with legacy automation need to seek out and deploy grab-and-go AI solutions, or else they stand the risk of being outpaced by smaller and more agile software companies with custom-built solutions.\n","date":"25 January 2022","permalink":"/writing/post-3/","section":"Curated List of Thoughts","summary":"Companies with legacy automation need to seek out and deploy grab-and-go AI solutions","title":"Grab and Go Edge-AI for Industry 4.0"},{"content":"","date":null,"permalink":"/tags/industry-4.0/","section":"Tags","summary":"","title":"Industry 4.0"},{"content":"","date":null,"permalink":"/tags/llm/","section":"Tags","summary":"","title":"LLM"},{"content":"","date":null,"permalink":"/tags/machine-learning/","section":"Tags","summary":"","title":"Machine Learning"},{"content":"","date":null,"permalink":"/tags/manufacturing/","section":"Tags","summary":"","title":"Manufacturing"},{"content":"","date":null,"permalink":"/tags/music/","section":"Tags","summary":"","title":"Music"},{"content":" Music is complex and varies widely in beats, frequencies and structure. This experiment uses the spectrogram of common songs to organize thousands of everyday tracks. We implement various neural network models to train classifiers to segregate neighboring songs with similar features from the mel-spectrogram of the songs. The softmax output of the neural network thus trained is used as a reduced feature set in order to perform dimensionality reduction of the songs. We then create different 2D embeddings of the reduced feature set using various dimensionality reduction techniques. These embeddings are used to place similar song features closer together on a visually interactive song-map. This, in our research so far, is the first attempt at producing a visual and interactive map of music at a full song collection scale that users can use in order to explore the world of music, discover and listen to songs of their preference.\n1. Methodology #In order to achieve the goal of scalability classifying thousands of songs while not loosing its transitive nature, our ap-proach does not perform a one-hot classification on the data. We also develop a means to reduce the size of the musicalfeatures from the raw audio of the song using mel-spectogram. We then train a neural network classifier on the reduced dataand use the softmax output in order to perform the visualization. The methodology we used is detailed below.\n1.1. Audio Feature Extraction using Mel-Spectogram #This process converts the song file into a 1366 x 966 dimension vector. In the pre-processing step, Mel-Spectrograms aregenerated in a similar manner to [11] whereby we use 96 mel-bins with 256 frame hops at sampling rate of 12000Hz, withthe Fast Fourier Transform window size of 512 frames around each frame. For the purposes of this project, we use 30 secondclips of songs in order to reduce computation time. Since the exact multiple of audio frames given the above hops,windowsand sampling rates for 30 seconds, comes out to be exactly 29.12 seconds, we trim 0.44 seconds of audio from the first andthe last portion of the clip. We use the audio processing tool developed by McFee et al [25] in order to perform the requiredmanipulation on the song data. The output of the pre-processing step is a song vector of dimension 1366 x 96 x (total numberof songs) which we feed into the neural network model in the subsequent section.\nFigure 1: Mel-Spectogram of a song\n1.2. Song Feature Extraction and Classification #The first step towards generating the map of songs is to extract meaningful features out of them. These would be featureslike song genre, beats, timbre, language, artists etc. The raw data of the song and its meta-data would be fed into a neuralnetwork and trained using classification labels available. In order to do this we used convolutional recurrent neural networl toclassify the songs into genres. We then looked for interesting patterns in the outputs of the softmax layer. This approach hasalready been tried out in neuroscience and vision [29] and we believe using the flattened features of the softmax/penultimatelayers of neural networks in music will also provide us with a reduced feature set that can be fed into our dimensionalityreduction later on. In order to test this approach, we generate a t-SNE plot directly of the Mel-spectogram of songs. Figure[3] shows that the raw Mel-spectogram does not provide us with a meaningful clustering of songs.\nFigure 2: Performing T-SNE of the mel-spectograms of the songs directly does not lead to distinct or distinguishable results\nFigure 3: Performing T-SNE of the mel-spectograms of the songs using the softmax output results in better classification and clustering\nFigure 4: Final CRNN model that gives 80% accuracy on test data\nThus, in order to achieve meaningful dimensionality reduction, the song features are decomposed into a 10 dimensionalvector using a convolutional recurrent neural network. We experimented with the size and depth of the neural network inorder to develop a suitable classifier for genre prediction, with a targeted accuracy of atleast 0.5. We tried models from [11]and [?] and finally built our own model. Using the right features are very important to identify genres [38] or in our casethe right feature vectors of the songs. Here we need to balance out the specificity, ie. response to particular type of songand the generality, ie. the amount of spread of feature vectors when the songs are fed into the system, so that we get adecent but relevant spread in our final song-map. If the outputs of the song classifier are too specific, the final dimensionalityreduction will have a lot of data points clustered close together and there will not be sufficient scatter in the data-points inthe visualization. On the other hand data that is poorly classified will have a lot of intermixed points which will lead to poor relevance of one song to other. Thus the aim of the project is not to exceed the performance metric of the existingclassification systems, but to tune the results of the neural networks to generate an appropriate set of feature vectors, whichupon being fed into the dimensionality reduction system provides us with a map of music which will be relevant to the user.\nThe final model we developed was a 8 layer Gated Recurrent Convolutional Neural Network with 6 convolutional layersand 2 recurrent layers. The convolutional layers extract high level features from the neural network while the recurrent layersprovide the ability to capture time-series information from the data. In this manner, features such as beats, treble, and vocalscan be thought of as a series of neuron activations in a sequential manner, with the convolutional layers aggregating lowerlevel features into higher level features, and the recurrent units firing, if a particular high-level feature is sustained for a fixedperiod of time, for example legato, staccato and vibrato in vocals[1]. Paper [6] explains some of these patterns in violinmusic.Even after mel-spectogram reduction, the size of the input was very large. Thus we optimized the algorithms in orderto reduce the computation time. We converted the tensorflow backend of keras to GPU based code in order to reducecomputation time from approximately 5 minutes for 1 epoch of 1000 songs, to around 40 seconds per epoch for 1000 songs.This was done by lowering the batch size and running the computation using an Nvidia GTX1080x GPU. This allowed us toscale up the data collection from 400 to 1000 to upto 23,000 songs.A table detailing the architecture of various models and their classification accuracy, with respect to various hyper-parameters are given.\nModel Dataset Epochs Training Accuracy Validation Accuracy Test Accuracy 4 Conv Blocks GTZAN 40 0.87 0.80 0.78 4 Conv Blocks GTZAN 300 0.89 0.78 0.75 6 Conv Blocks GTZAN 100 0.93 0.65 0.60 6 Conv Blocks GTZAN + FMA 100 0.90 0.80 0.80 Table 1: Accuracies for different neural network models.\n1.3. Visualization #There are multiple methods to perform dimensionality reduction. Principal Component Analysis [36] and Linear Discrim-inant Analysis [19] are two commonly used methods to reduce multidimensional features into two or three dimensions.PCAprovides an intuitive understanding of the music map for a starting point.In our experiment(Fig. 6(a)),the transitions of var-ious genres of music form two main outward branches, one being Metal and the other being Hip-hop. Pop music graduallytransitions into hip-hop, while rock music transitions into metal. Classical, country and reggae music appear as three branchesforming the base of the Y stem which merges in the center. This trend is still valid on music on 1000 song dataset or the23,000 song dataset.\nHowever, PCA as shown in Fig 6.b also indicates that the map becomes less intuitive as the dataset size increases and thenumber of available songs in one or two genres have a larger majority than the other. Thus PCA, though providing an initialunderstanding of the way music is comprised, does not scale well to large datasets due to its many inherent limitations.Forexample, PCA is a bad choice because it is a linear algorithm which cannot distinguish non-linear structures in the data. Anin-depth study of these limitations is performed by [28], albeit for a different use case. Hence we are exploring the use oft-SNE [24] and UMAP [26] in order to develop the music map.These techniques perform much better than PCA for largesongs, as can be seen in the results section(Figure 10).\nFigure 6:\n(a) PCA on GTZAN dataset of1000 songs\n(b) PCA on GTZAN+FMAdataset of 23,141 songs\nPCA plots of 10 different genres of music show that this method is not scalable, although it is easy to comprehendthe transition between different genres of music.\n1.4. User Interface #Finally we use the music map generated by the visualization into an interactive user interface that allows the user to clickand play the songs in the area selected. To accomplish this task, we used the popular web browser rendering tool Three.js[14] in order to render the songs. The input to the web application is a json file of coordinates and paths of each song in anested list dictionary structure. These ’embeddings’ are generated for all the songs present in the database, in order to getthe best possible visual output. The points encode a GET request to the URL of the song in the database, which is playedback using the Web Audio API [10]. We also used another extension to Web Audio API called webaudiox.js in order to addadditional functionality such as audio volume normalization, crossfading and sound smoothing in order to make the songsmore appealing to the listener. Note that all songs of a particular genre have the same colour. The web page was designed using a template from [4].\nFigure 7: User interface of our application\n2. Datasets Used #There are millions of songs in the world to chose from and add to our system. With an impartial feature classifier, we hopeto get better and better at mapping songs to the point where any new song released can be added to the system and it will putthe song into the right spot on our map. However, given the limitations of time and computing power, we start with smallersong sets and learn the features on smaller subsets of music. Incrementally, we worked on the following datasets :\nGTZAN Dataset : This dataset is the most-used public dataset for evaluation in machine listening research for music genre recognition (MGR). It was first used for the well known paper in genre classification [33]. The dataset consists of 1000 audio tracks each 30 seconds long distributed among 10 genres, each represented by 100 tracks. The tracks are all 22050Hz Mono 16-bit audio files in .wav format. dataset clips genres length[s] size[GiB]\nsmall 8,000 8 30 7.6 medium 25,000 16 30 23 large 106,574 161 30 98 full 106,574 161 278 917\nTable 2: FMA data subsets\nFigure 8: Distribution of no.of tracks per Genre in FMA dataset\nFigure 9: Distribution of no.of tracks per Genre used in our music map after filtering\nFree Music Archive Dataset: We wanted to train our Neural network model on relatively bigger dataset than GTZAN. Although Million Song Dataset (MSD)[7] as well as the newer AudioSet and AcousticBrainz are very large-scale reference datasets, they do not provide raw mp3 files and downloading the mp3 files in itself is a challenge due to copyright issues. FMA provides an excellent alternative as a medium scale music dataset of more than 100k songs with freely available full-length and high-quality audio. It provides pre-computed features, together with track- and user-level meta-data, and tags. The dataset is divided and maintained in subsets listed in Table 2.The problem with the FMA dataset was that there was a non-uniform distribution of songs in different genres(Figure 8).Hence, we merged both GTZAN and FMA datasets together and selected 10 popular genres and filtered the songs to be usedfor our visualization. Distribution of trakcs per genre was still dominated by few genres but it has helped us further ouranalysis on Genres selected from GTZAN dataset (Figure 9). Finally, the filtered dataset consisted of 23,141 songs whichwere divided among training testing and validation set, with 14,809 songs for training, 3,703 for validation and 4,629 fortesting. 3. Results #Plotting songs on a 2D map gave us many auditory insights which are hard to describe on paper and vary from person toperson. On the other hand, visual properties are easier to describe in general. For example, the t-SNE algorithm has a hyper-parameter - Perplexity, which balances attention between local and global structures of the data. It specifies the number ofclose neighbors each point has. We observed that as we increase the perplexity value we started getting more visually distinctclusters of music genres.UMAP has two hyper-parameters: no-of-neighbors, which indicates the number of neighboring points in the local ap-proximations of the manifold structure. If more neighbors are considered, lower dimension preserves global structure in dataset.\nThe second parameter is min-dist, which controls how tightly the embedding is allowed to compress points together.If compressing is allowed with higher margin it ensures that the embedded points are more evenly distributed and retain theglobal structure in dataset.It was observed that with larger values of both no-of-neighbors and min-dist we started getting much better clusters ofmusic genres on our dataset. In conclusion, we found that the distribution of points by UMAP is more visually appealing thant-SNE to most viewers. Figure 10 shows the various outputs generated by the system for varying parameter combinations.\nFigure 10: Music Maps generated using different t-SNE and UMAP parameters\n4. Conclusion #We present a novel method of visualizing music using the latest state-of-art visualization techniques, powered by a custombuilt neural network that matches the current state-of-art in classification accuracy, and is optimized to run an epoch of 1000songs in under a minute. This enables us to develop the first mapping of music on a full-song collection level in currentliterature, to the best of our knowledge. The above methodology is successfully demonstrated on a dataset of 23,000 audiotracks which demonstrates the viability of the idea that the softmax output of the neural network can be used as a reducedfeature set for song visualization. The outputs of the visualization algorithms, are not only mysterious but also beautiful,presenting the audience with an unprecedented way of understanding music, opening up a new dimension in how music is perceived and on a larger scale, interacted with. This, we believe is the greatest outcome of the project.\n5. References #[1] 3 voice techniques: Legato, staccato, vibrato: https://www.youtube.com/watch?v=shtz01-zie8.\n[2] Data to date: the rapid rise of social and streaming: https://www.nextbigsound.com/industry-report/2015.\n[3] Google infinite drum machine: https://experiments.withgoogle.com/ai/drum-machine.\n[4] Umap tsne embedding visualiser: https://github.com/fedden/umap-tsne-embedding-visualiser.\n[5] G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possibleextensions.IEEE transactions on knowledge and data engineering, 17(6):734–749, 2005.\n[6] L. Aversano. Terminologia violinistica tra sei e settecento.Tra le note, pages 1000–1034, 1996.\n[7] T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere. The million song dataset. InIsmir, volume 2, page 10, 2011.\n[8] K. Bunte, M. Biehl, and B. Hammer. A general framework for dimensionality-reducing data visualization mapping.Neural Compu-tation, 24(3):771–804, 2012.\n[9]O. Celma, M. Ramirez, and P. Herrera. Foafing the music: A music recommendation system based on rss feeds and user preferences.Inin ISMIR. Citeseer, 2005.\n[10] H. Choi and J. Berger. Waax: Web audio api extension. InNIME, pages 499–502, 2013.\n[11] K. Choi, G. Fazekas, M. Sandler, and K. Cho. Convolutional recurrent neural networks for music classification. InAcoustics, Speechand Signal Processing (ICASSP), 2017 IEEE International Conference on, pages 2392–2396. IEEE, 2017.\n[12] M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson. Fma: A dataset for music analysis.arXiv preprint arXiv:1612.01840,2016.\n[13] S. Dieleman and B. Schrauwen. Multiscale approaches to music audio feature learning. In14th International Society for MusicInformation Retrieval Conference (ISMIR-2013), pages 116–121. Pontifıcia Universidade Catolica do Parana, 2013.\n[14] J. Dirksen.Learning Three. js: the JavaScript 3D library for WebGL. Packt Publishing Ltd, 2013.\n[15] P. Filzmoser, K. Hron, and C. Reimann. Univariate statistical analysis of environmental (compositional) data: problems and possibil-ities.Science of the Total Environment, 407(23):6100–6108, 2009.\n[16] S. Flesch, A.-S. Gutsche, and D. Paschen. Exploring the untapped potential of sound maps.\n[17] J. Foote. Visualizing music and audio using self-similarity. InProceedings of the seventh ACM international conference on Multime-dia (Part 1), pages 77–80. ACM, 1999.\n[18] W. Hill, L. Stead, M. Rosenstein, and G. Furnas. Recommending and evaluating choices in a virtual community of use. InProceedingsof the SIGCHI conference on Human factors in computing systems, pages 194–201. ACM Press/Addison-Wesley Publishing Co.,1995.\n[19] A. J. Izenman. Linear discriminant analysis. InModern multivariate statistical techniques, pages 237–280. Springer, 2013.\n[20] F.-F. Kuo, M.-F. Chiang, M.-K. Shan, and S.-Y. Lee. Emotion-based music recommendation by association discovery from filmmusic. InProceedings of the 13th annual ACM international conference on Multimedia, pages 507–510. ACM, 2005.\n[21] S. W. Lee, J. Bang, and G. Essl. Live coding youtube: Organizing streaming media for an audiovisual performance.Ann Arbor,1001:48109–2121, 2017.\n[22] M. Levy and M. Sandler. A semantic space for music derived from social tags.Austrian Compuer Society, 1:12, 2007.\n[23] T. Li, M. Ogihara, and Q. Li. A comparative study on content-based music genre classification. InProceedings of the 26th annualinternational ACM SIGIR conference on Research and development in informaion retrieval, pages 282–289. ACM, 2003.\n[24] L. v. d. Maaten and G. Hinton. Visualizing data using t-sne.Journal of machine learning research, 9(Nov):2579–2605, 2008.\n[25] B. McFee, C. Raffel, D. Liang, D. P. Ellis, M. McVicar, E. Battenberg, and O. Nieto. librosa: Audio and music signal analysis inpython. InProceedings of the 14th python in science conference, pages 18–25, 2015.\n[26] L. McInnes and J. Healy. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprintarXiv:1802.03426, 2018.\n[27] H.-S. Park, J.-O. Yoo, and S.-B. Cho. A context-aware music recommendation system using fuzzy bayesian networks with utilitytheory. InInternational conference on Fuzzy systems and knowledge discovery, pages 970–979. Springer, 2006.\n[28] S. Prasad and L. M. Bruce. Limitations of principal components analysis for hyperspectral target recognition.IEEE Geoscience andRemote Sensing Letters, 5(4):625–629, 2008.\n[29] S. Rohit and S. Chakravarthy. A convolutional neural network model of the neural responses of inferotemporal cortex to complexvisual objects.BMC neuroscience, 12(1):P35, 2011.\n[30] S. Saxena and C. J. Romanowski. Theme extraction from lyrics.\n[31] Y. Song, S. Dixon, and M. Pearce. A survey of music recommendation systems and future perspectives. In9th InternationalSymposium on Computer Music Modeling and Retrieval, volume 4, 2012.\n[32] M. Torrens, P. Hertzog, and J. L. Arcos. Visualizing and exploring personal music libraries. InISMIR, 2004.\n[33] G. Tzanetakis and P. Cook. Musical genre classification of audio signals. IEEE Transactions on speech and audio processing,10(5):293–302, 2002.\n[34] A. Van den Oord, S. Dieleman, and B. Schrauwen. Deep content-based music recommendation. InAdvances in neural information processing systems, pages 2643–2651, 2013.\n[35] M. M. Van Hulle. Self-organizing maps. InHandbook of Natural Computing, pages 585–622. Springer, 2012.\n[36] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis.Chemometrics and intelligent laboratory systems, 2(1-3):37–52,1987.\n[37] Y. Xia, L. Wang, and K.-F. Wong. Sentiment vector space model for lyric-based song sentiment classification.International Journalof Computer Processing Of Languages, 21(04):309–330, 2008.\n[38] E. Zheng, M. Moh, and T.-S. Moh. Music genre classification: A n-gram based musicological approach. InAdvance ComputingConference (IACC), 2017 IEEE 7th International, pages 671–677. IEEE, 2017.\n","date":null,"permalink":"/projects/music-map/","section":"Projects","summary":"Learn more about the first attempt at producing a visual and interactive map of music at a full song collection scale.","title":"Music-Mapp : The world's first map of music"},{"content":"AI Safety, Generative AI, Robotics, Machine Learning, Unmanned Systems, Music\n","date":null,"permalink":"/projects/","section":"Projects","summary":"AI Safety, Generative AI, Robotics, Machine Learning, Unmanned Systems, Music","title":"Projects"},{"content":"Coding is the bread and butter for many software engineers in the highly competitive tech industry. However not many developers invest sufficient time towards sharpening their programming skills once they land a tech job past their interviews. While this article might not inherently make a developer a better programmer, I am assured by my experience in the industry that the following tips will definitely help any developer become faster at their day to day job.\nTip 1 : Master the art of Command line editing. #While some developers assume that using tools such as emacs or vim often fall into the realms of legendary code ninjas and hackers, the opposite is actually true. An average developer takes approximately 1-3 seconds to transition from the keyboard to the mouse, and roughly 1 second per click on a GUI element for purposes such as commiting and pushing code. These few seconds, while insignificant in the span of a day\u0026rsquo;s work, eventually add up to over 30 minutes in a workday, or 2 and a half hours in a week, or about 150 hours in a year. That\u0026rsquo;s roughly more than a week of extra time wasted just transitioning between the keyboard and the mouse.\nNow imagine if you could reduce this time? How do you do it?\nHere are some steps.\nSet the three fingered salute : Ctrl + Alt + T to pop open a terminal if your system doesn\u0026rsquo;t already do so.\nGet used to simple command line operations such as cp, cd, ls, pwd, !, and the git commands. (more on git later..)\nLearn the habit of using Tab key to autocomplete file paths and directory folders and basic command line tools.\nLearn an easy to use text editor so you can quickly hop into a file and edit it without leaving the keyboard. This might seem hard at first, but over time it get\u0026rsquo;s easier. Trust me. Here\u0026rsquo;s a short intro to vi with the three commands I use the most.\nuse I to get into editing ( insert mode) use Esc + :wq! to write your changes, quit and exit use Esc + :q! to quit without editing your changes. You can learn more commands over time to get better at it, but the above three are good enough to get you started. Here\u0026rsquo;s a snippet from my terminal as I publish this article.\nTip 2 : Learn Git, on the command line. #VSCode and other editors often lull you into complacency via the easy to use commit and push buttons, but the downside of using these is that you effectively become slower in making changes and often you\u0026rsquo;re left completely clueless on what to do when the UI breaks for your use case.\nOn the other hand, learning git on the CLI enables you to work faster, by making branches on the fly, and trying out code, reverting, etc. The key here is to become really good at failing fast, and trying out new directions faster. I often make 10\u0026rsquo;s of branches for a project locally and just delete them on the fly if my line of code does not work in the branch, effectively checkpointing my development and failing and iterating quickly. To aid with this, I often create shortcuts to use git on the terminal. Here are some of them.\nalias gc=\u0026#34;git commit\u0026#34; alias gs=\u0026#34;git status\u0026#34; alias gad=\u0026#34;git add .\u0026#34; alias gamend=\u0026#34;git commit --amend\u0026#34; alias gp=\u0026#34;git push\u0026#34; alias gpf=\u0026#34;git push -f\u0026#34; alias squasher=\u0026#39;git reset $(git merge-base main $(git rev-parse --abbrev-ref HEAD)) \u0026amp;\u0026amp; git add -A \u0026amp;\u0026amp; git commit\u0026#39; gcmsg() { echo Commiting : $1 git add . \u0026amp;\u0026amp; git commit -am $1 } For those who are completely clueless and new to git, here\u0026rsquo;s a series I highly reccomend watching. The video shows how prevalent and useful learning git is for anyone who uses a computer, from a software engineer to poets\nGit and Github for Poets ( source: The Coding Train)\nTip 3: Aliases: The secret to programming 2x faster on the terminal #Create a file called aliases in your $HOME or root directory. In your shell\u0026rsquo;s .rc file ( eg .bashrc, or .zshrc ) add a line at the very end : source $HOME/aliases.\nThis will ensure that every time your terminal loads, your shortcuts are loaded. Finally inside your aliases file, create shortcuts for frequently used commands.\nHere are some aliases I use on a daily basis. Let\u0026rsquo;s go through some of them.\n# Exports # All useful export variables go here. For example export TEMP=$HOME/my/temp # Shortcuts alias aliases=\u0026#39;code $HOME/aliases\u0026#39; alias reload=\u0026#39;source $HOME/aliases \u0026amp;\u0026amp; echo Reloaded Aliases!\u0026#39; alias Dls=\u0026#34;cd $HOME/Downloads\u0026#34; alias Docs=\u0026#34;cd $HOME/Documents\u0026#34; alias pgweb=\u0026#39;docker run --network=host sosedoff/pgweb\u0026#39; alias schemaspy=\u0026#39;mkdir -p diagram/output/tables \u0026amp;\u0026amp; docker run --network=host -v `pwd`/diagram:/output schemaspy/schemaspy:snapshot -t pgsql -host 0.0.0.0 -u postgres -db postgres -p pass\u0026#39; alias leetcode=\u0026#39;google-chrome https://leetcode.com/ \u0026amp;\u0026amp; code $HOME/Personal/Leetcode\u0026#39; alias rcp=\u0026#34;rsync -ah --progress\u0026#34; alias bashrc=\u0026#34;code $HOME/.bashrc\u0026#34; alias cleanpy=\u0026#34;sudo find . | grep -E \u0026#39;(__pycache__|\\.pytest_cache|\\.pyc|\\.pyo$)\u0026#39; | xargs sudo rm -rf\u0026#34; alias killdoc=\u0026#39;docker rm -f $(docker ps -q)\u0026#39; alias gfmt=\u0026#39;gofmt -w .\u0026#39; killport() { echo Killing port $1 lsof -ti:$1 | xargs kill } # Git alias gc=\u0026#34;git commit\u0026#34; alias gs=\u0026#34;git status\u0026#34; alias gad=\u0026#34;git add .\u0026#34; alias gamend=\u0026#34;git commit --amend\u0026#34; alias gp=\u0026#34;git push\u0026#34; alias gpf=\u0026#34;git push -f\u0026#34; alias squasher=\u0026#39;git reset $(git merge-base main $(git rev-parse --abbrev-ref HEAD)) \u0026amp;\u0026amp; git add -A \u0026amp;\u0026amp; git commit\u0026#39; gcmsg() { echo Commiting : $1 git add . \u0026amp;\u0026amp; git commit -am $1 } # Flatten Directory flatten() { find . -type f -print0 | xargs -0 -I file mv --backup=numbered file . find . -type d -empty -delete rename \u0026#39;s/((?:\\..+)?)\\.~(\\d+)~$/_$2$1/\u0026#39; *.~*~ } #Python alias cleanpy=\u0026#34;sudo find . | grep -E \u0026#39;(_pycache_|\\.pytest_cache|\\.pyc|\\.pyo$)\u0026#39; | xargs sudo rm -rf\u0026#34; alias venvs=\u0026#34;cd $HOME/.venv/\u0026#34; source $HOME/.venv/virtualenvs create-venv() { if [ $# -eq 0 ] then echo \u0026#34;Please provide \u0026lt;venv-name\u0026gt; in order to create the virtual environment\u0026#34; fi if ! [[ $1 =~ ^[0-9a-zA-Z_]+$ ]]; then echo \u0026#39;Note that only a-z, A-Z, 0-9 and underscore characters are allowed in \u0026lt;venv-name\u0026gt;\u0026#39; \u0026gt;\u0026amp;2 # write to stderr return 1 fi python3 -m venv $HOME/.venv/$1 echo Created Virtual Env : $1 echo \u0026#34;alias activate-$1=\\\u0026#34;source $HOME/.venv/$1/bin/activate\\\u0026#34;\u0026#34; | tee -a $HOME/.venv/virtualenvs \u0026gt;/dev/null source $HOME/.venv/virtualenvs source $HOME/.venv/$1/bin/activate } list-venvs() { cat $HOME/.venv/virtualenvs | sed -e \u0026#39;s/.*alias activate-\\(.*\\)=.*/\\1/\u0026#39; } remove-venv() { target=$1 read -q \u0026#34;REPLY?Remove virtualenv $target? (y/n):\u0026#34; echo if [[ $REPLY =~ ^[Yy]$ ]] then rm -rf $HOME/.venv/$target sed -i \u0026#34;/$target/d\u0026#34; $HOME/.venv/virtualenvs echo \u0026#34;Virtualenv $target removed!\u0026#34; ","date":"25 January 2022","permalink":"/writing/post-4/","section":"Curated List of Thoughts","summary":"A collection of time saving tips and tricks I have gathered over the years, that helps me develop code 2X faster than many peers.","title":"Tips and tricks to code 2x faster"},{"content":"","date":null,"permalink":"/projects/robust-ml/","section":"Projects","summary":"","title":""},{"content":" Over the span of more than 10 years in the industry, I have gained expertise in various facets of applied Artificial Intelligence, especially around Security and Scalablity of AI in Cloud Computing, Generative AI and Unmanned Systems. I have been fortunate to work on some of the most cutting-edge technology in engineering, an overview of which can be found below.\nNVIDIA : AI Safety, Gen-AIJun 2023 - Present\n# Building state-of-the-art Generative AI products under NVIDIA Picasso Edify. Responsible for AI Safety and Security of the products, and a founding member of the Red Team 🚩 efforts to ensure safe, unbiased and trustworthy AI.\nDeployed text-to-image, image-to-image, text-to-3d, image-to-3d AI models in partnership with Getty Images and led research-to-production pipeline bring-up to enable continuous deployment of the latest AI models and checkpoints, with a turnaround time of less than 3 hours.\nI also mentor junior engineers and conduct technical interviews for new hires.\nHighlights ⭐ Try our generative AI offerings below ⭐\nText to Image Image to 3D Text to 360 NVIDIA : Self-Driving CarsOct 2018 - Jun 2023\n#Developed hyper scale simulation and replay capabilities for AV Verification, Validation, and iterative improvement for autonomous vehicles. Built software that helps self-driving cars become safer by training AI to drive billions of miles in photorealistic virtual worlds. Bringup of hardware-in-the-loop simulation on Nvidia DriveConstellation servers.\nETH Zürich - RoboticsSep 2018\n#Instructor for Robotic Architecture at Rob-Arch 2018 and the preparation leading up to it. Taught the theoretical and practical knowledge required for closed loop controlled robot 3D Printing on arbitrary surfaces, using ROS industrial, Move-it, RViz and a custom built sensor driver package for the Wenglor Laser scanner.\nKUKA - Cloud IntelligenceJun 2018 - Aug 2018\n#Developed drivers to interface Roboception GmbH’s 3d stereo-vision camera to robot operational data stream and provide stereo images to the AI module • Full stack cloud intelligence software development for Kuka Connect• Developed the first Windows Linux Hybrid Kubernetes cluster using ’Top of Rack’ Routing on AWS at KUKA\nTexas A\u0026amp;M Transportation Institute - AI ResearchOct 2017 - May 2018\n#Conducted research with Prof. Srikant Saripalli to analyse classifier performance for Traffic Sign Detection. Performed PCA and t-SNE analysis to study in-class variation of traffic signs installed across the state of Texas. Developed an ultra-high-speed sign detection \u0026amp; classification framework for traffic signs inspired by research on kernelized correlation filters from Visual Geometry Group, University of Oxford. Current results : 350 fps\nDassault Systèmes - SimulationJun 2016 - Jul 2017\n#Worked in the R\u0026amp;D team of Dassault Systèmes DELMIA on kinematics and motion planning for industrial robots • Developed a new simulation player that is still in use today. Achieved 4x performance improvements by optimizing multi-threaded C++ code of the simulation player\nBajaj Auto - Industrial Robotics \u0026amp; AIJul 2014 - Jun 2016\n#Part of team of 12 at the world\u0026rsquo;s largest 3 wheeler automaker, handling the entire end-to-end automation effort within Bajaj Auto\u0026rsquo;s manufacturing sector. Introduced Sync-Slide robot, a collaborative robotic bolt tightening system with image processing, force feedback and selective compliance.\nSync Slide was featured by Universal Robotics as a flagship project worldwide and a case study in women empowerment. Also developed an in-house IOT based real-time assembly line monitoring and predictive analysis system with 1000+ data points/sec from 5000 sensors.\nIndian Institute of Science - Swarm IntelligenceMay 2013 - Jul 2013\n#Working with Prof. Debashish Ghose I developed algorithms for Path Planning for Quadcopters to Survey Damaged Building Area. The relevance of this project has had a deeper impact on me, following the 2013 Kedarnath floods. These algorithms enable disaster response teams to rapidly deploy a team of various autonomous quadcopters over the disaster scenario to identify and locate survivors within the first few hours of the disaster, so that rescue teams could directly \u0026lsquo;approach\u0026rsquo; the location of the trapped survivors, without having to spend valuable time and effort searching for them. My work lays down the foundation strategies and mathematical algorithms to achieve this end.\nRobotics Interest Group - Co-FounderOct 2011 - present\n#Co-Founded Robotics Interest Group, the first academic undergraduate research body in National Institute of Technology - Calicut. Led the development and integration of an indigenous 5 DOF mobile manipulator, Speech Recognition based Interactive Robot, High Voltage Transmission Line Inspection Robot and Bio-mimetic simulations. Robotics Interest Group is presently Kerala\u0026rsquo;s most advanced academic robotics group. Presently serving as the honorary member and advisor to the team.\n","date":null,"permalink":"/career/","section":"Jibin Rajan Varghese","summary":"Over the span of more than 10 years in the industry, I have gained expertise in various facets of applied Artificial Intelligence, especially around Security and Scalablity of AI in Cloud Computing, Generative AI and Unmanned Systems.","title":"Career"},{"content":"","date":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories"},{"content":"Research Interests #AI Safety, Generative AI, Robotics, Machine Learning, Unmanned Systems, Music\nPublications #[1] Varghese, Jibin, et al. \u0026ldquo;Disaster management using unmanned aerial vehicles.\u0026rdquo; Unmanned Aerial Systems. Academic Press, 2021. 129-155.\n[2] Baig, Mohammed Habibullah, Jibin Rajan Varghese, and Zhangyang Wang. \u0026ldquo;Musicmapp: a deep learning based solution for music exploration and visual interaction.\u0026rdquo; Proceedings of the 26th ACM international conference on Multimedia. 2018.\nPatents #A Radial Air Actuated Robotic Gripper Attachable to A Robot Manipulator ArmIndian Patent No: 375918\nRobot for High Voltage Electrical Transmission Line InspectionIndian Patent No: 481420\nEducation #Texas A\u0026M UniversityMaster of Computer Science\nNational Institute of Technology CalicutB-Tech in Mechanical Engineering\n","date":null,"permalink":"/research/","section":"Jibin Rajan Varghese","summary":"Research Interests #AI Safety, Generative AI, Robotics, Machine Learning, Unmanned Systems, Music","title":"Research"}]